A 2D Gabor-wavelet baseline model out-performs a 3D surface model in scene-responsive cortex

Abstract

Understanding 3D representations of spatial information, particularly in naturalistic scenes, remains a significant challenge in vision science. This is largely because of conceptual difficulties in disentangling higher-level 3D information from co-occurring features and cues (e.g., the 3D shape of a scene image is necessarily defined by "low-level" spatial frequency and orientation information). Recent work has employed newer models and analysis techniques that attempt to mitigate these difficulties within a model-comparison framework. For example, one such study reported 3D-surface features were uniquely present in areas OPA, PPA, and MPA/RSC (areas typically referred to as scene-selective), above and beyond a Gabor-wavelet baseline model. Here, we tested whether these findings generalized to a new stimulus set that, on average, dissociated static Gabor-wavelet baseline features from 3D scene-surface features. Surprisingly, we found evidence that a Gabor-wavelet baseline model--commonly thought of as a "low-level" or "2D" model--better fit voxel responses in areas OPA, PPA and MPA/RSC compared to a model with 3D-surface information. We highlight that this difference in results could be due to differences in the baseline conditions used across studies. These findings emphasize that much of the information in "scene-selective" regions--potentially even information about 3D surfaces--may be in the form of spatial frequency and orientation information often considered 2D or low-level. Disentangling lower-level and higher-level visual information is a continuing fundamental challenge for model-comparison approaches in visual cognition, and it motivates future work investigating which visual features could cue higher-level properties in our real-world visual experience--both within and beyond current model comparison frameworks. Author summaryTo gain a more complete picture of human visual processing, it is critical to understand the precise format of representations of naturalistic visual scenes. Recent work has approached this challenge by quantifying how much of our brain activity might be due to hypothesized characteristics of the stimuli being viewed. Here, we followed up on work finding that activity in scene-responsive regions of the brain is well predicted by information about the 3D configurations of major surfaces in viewed scenes, like walls and floors. In contrast to previous work, we found that our baseline condition--commonly thought of as "low-level" visual information--accounted for responses in these regions better than both of the 3D surface models that we tested. We highlight that this difference in results could be due to differences in the baseline conditions used across studies. However, our findings do not necessarily argue against the importance of these regions in encoding 3D surface information. Instead, they highlight the possibility that baseline models can perform well by virtue of covariation between low-level features like orientation and spatial frequency with higher-level properties like depth information. This motivates future work and/or new analysis frameworks to better characterize the interplay between hypothesized model features and the specific techniques used to quantify their relationship to neural representations.