Wundari, B. G. | Search bioRxiv

bioRxiv2026

Correlation-based binocular disparity computations induce representational bottlenecks at the population level

Binocular stereopsis depends on comparing the images seen by the two eyes. Although correlation-based models explain responses of individual binocular neurons in primary visual cortex (V1), it remains elusive whether population representations formed by local correlation activity patterns can support depth perception under ambiguous inputs. Using psychophysics, fMRI, and neural network modeling, we tested human stereopsis with dynamic anticorrelated stimuli that were dominated by binocular mismatches. Humans reliably perceived reversed depth as predicted by correlation-based computations, yet population representations consistent with this percept were detected in V3A, not V1. Shallow and deep neural networks constrained by correlation-like binocular interactions did not capture the full pattern of human depth judgments. Analyses of their internal representations showed greater representational overlap, whereas deep architectures not constrained to explicit correlation interactions exhibited less entangled representations and better aligned with human behavior. These findings suggest that biological stereopsis may rely on population coding beyond correlation-like computations. Significance StatementThe brain must infer depth from binocular inputs that are inherently ambiguous. Although correlation-based models explain disparity tuning of individual neurons in primary visual cortex (V1), whether these local mechanisms support perceptual inference at the population level remains unclear. Using psychophysics, fMRI, and neural network modeling, we show that population representations consistent with perceived depth under ambiguity were detected in mid-dorsal area V3A, not V1. Analyses of how neural networks encode multiple features showed that correlation-based computations represent features into overlapping activity patterns that may constrain downstream readout and degrade depth estimates. In contrast, models not constrained by explicit interocular correlation maintained more distinct population codes and closely matched human perception. These findings suggest that architectures constrained to explicit correlation-like processing can form population representations that are suboptimal to explain human stereopsis, motivating hybrid mechanisms.

neuroscience↗

bioRxiv2024

Correlation and Matching Representations of Binocular Disparity across the Human Visual Cortex

Seeing three-dimensional objects requires multiple stages of representational transformation, beginning in the primary visual cortex (V1). Here, neurons compute binocular disparity from the left and right retinal inputs through a mechanism similar to local cross-correlation. However, correlation-based representation is ambiguous because it is sensitive to disparities in both similar and dissimilar features between the eyes. Along the visual pathways, the representation transforms to a cross-matching basis, eliminating responses to falsely matched disparities. We investigated this transformation across the human visual areas using functional magnetic resonance imaging (fMRI) and computational modeling. By fitting a linear weighted sum of cross-correlation and cross-matching model representations to the brains representational structure of disparity, we found that areas V1-V3 exhibited stronger cross-correlation components, V3A/B, V7, and hV4 were slightly inclined towards cross-matching, and hMT+ was strongly engaged in cross-matching. To explore the underlying mechanism, we identified a deep neural network optimized for estimating disparity in natural scenes that matched human depth judgment in the random-dot stereograms used in the fMRI experiments. Despite not being constrained to match fMRI data, the network units responses progressed from cross-correlation to cross-matching across layers. Activation maximization analysis on the network suggests that the transformation incorporates three phases, each emphasizing different aspects of binocular similarity and dissimilarity for depth extraction. Our findings suggest a systematic distribution of both components throughout the visual cortex, with cross-matching playing a greater role in areas anterior to V3, and that the transformation exploits responses to false matches rather than discarding them. Significant StatementHumans perceive the visual world in 3D by exploiting binocular disparity. To achieve this, the brain transforms neural representation from the cross-correlation of signals from both eyes into a cross-matching representation, filtering out responses to disparities from falsely matched features. The location and mechanism of this transformation in the human brain are unclear. Using fMRI, we demonstrated that both representations were systematically distributed across the visual cortex, with cross-matching exerting a stronger effect in cortical areas anterior to V3. A neural network optimized for disparity estimation in natural scenes replicated human depth judgment in various stereograms and exhibited a similar transformation. The transformation from correlation to matching representation may be driven by performance optimization for depth extraction in natural environments.

neuroscience↗