Decoding 3D spatial location across saccades in human visual cortex

Abstract

Visual signals are initially processed as two-dimensional images on our retina. In daily life, we make frequent eye movements and consequently the 2D retinal inputs constantly change. In addition, to perceive a 3D world, depth information needs to be reconstructed, using cues such as the binocular disparity between the 2D retinal images from both eyes. How do saccades influence the brain representation of 3D spatial locations? In an fMRI scanner, while wearing red-green anaglyph glasses to facilitate 3D perception, participants passively viewed a random dot patch that stimulated one of four 3D locations in each 16-second block. Each location was defined by its 2D position (above or below screen center; vertical information), and its depth position (in front of or behind central screen plane). We compared the amount of 2D and depth information (using multivariate pattern analysis) for no-saccade blocks (in which participants maintained stationary fixation) compared to saccade blocks (a series of guided saccades). In saccade blocks, we could decode vertical and depth information to a similar extent as in no-saccade blocks, despite the retinal changes in horizontal position induced by the saccades. Strikingly, no-saccade blocks exhibited a strong dependence on fixation position: little vertical or depth information could be decoded across blocks with different fixation positions in any visual areas during stable fixation. In contrast, on saccade blocks, both vertical and depth information were tolerant of changes in fixation position. The findings suggest that representations of 3D spatial locations may become more tolerant of fixation positions during "dynamic" saccades, perhaps due to active remapping which may encourage more stable representations of the world.