Unfolding spatiotemporal representations of 3D visual perception in the human brain

Abstract

Although visual input is initially recorded in two dimensions on our retinas, we perceive and interact with the world in three dimensions. Achieving 3D perception requires the brain to integrate 2D spatial representations with multiple depth cues, such as binocular disparity. However, most studies typically examine 2D and depth information in isolation, leaving the integrated nature of 3D spatial encoding largely underexplored. In this study, we collected a densely sampled multimodal neuroimaging dataset from 10 participants (8 with EEG and fMRI; 2 with fMRI only) across multiple sessions while they viewed stereoscopic 3D stimuli through red-green anaglyph glasses. Participants first completed a behavioral session including depth judgement tasks and a novel cube adjustment task to quantify and calibrate individual depth perception in units of binocular disparity. Then during two EEG and two fMRI sessions, participants passively viewed stimuli presented at 64 systematically sampled 3D locations, yielding over 66,000 trials in total across ten participants. Combining this multimodal dataset with computational methods via representational similarity analysis, we examined how 2D, depth-related, 3D feature-level, and geometric distance representations unfold across time and brain space. We found that the human brain represents 3D visual space not only by encoding position-in-depth as an additional dimension alongside 2D location, but also by constructing richer forms of 3D spatial structure. Specifically, 2D spatial features were represented earliest and most broadly, depth-related representations were weaker and more spatially restricted, and 3D feature representations were sparse and heterogeneous but detectable at the individual feature level. Critically, geometric distance analyses revealed that neural coding extended beyond separable feature dimensions, showing robust integrated 2D geometric representations and more selective evidence for integrated 3D geometric representations. These findings suggest that human 3D spatial perception is supported by a progression from dominant 2D coding to depth-related and 3D representations, with additional evidence for geometric structure in 3D space, contributing to a more comprehensive understanding of the spatiotemporal organization of neural representations that support 3D perception. Additionally, our novel large dataset will be made openly available to support future research on 3D perception and spatial cognition.