A tale of three types of internal object representations in the human brain

Abstract

Perception and mental imagery are widely thought to rely on overlapping neural representations, giving rise to perception-like visual experiences during imagery generation. However, such overlap, by itself, is agnostic to the nature of the underlying neural geometry for both perception and imagery. Here we addressed this question by comparing visualizers and individuals with congenital aphantasia using fMRI, representational similarity analysis, and multimodal deep neural network models to probe different types of neural geometry. Despite comparable behavioral performance and perception-imagery generalization of neural object representations in the ventral occipitotemporal cortex, the two groups exhibited systematically different object representational geometries already in the perceptual encoding phase. Visual-similarity-aligned geometry was selectively reduced in the left lateral occipitotemporal cortex of individuals with aphantasia during perceptual encoding, before imagery generation, and its expression during imagery covaried with imagery vividness. In contrast, language-relation-aligned geometry was preserved across groups during both perception and imagery. Together, these findings suggest that subjective imagery experience is associated with the geometry of neural object representations during both perception and imagery, and that representations within perception- imagery cross-decoding regions comprise heterogeneous components rather than a unitary shared neural code.