Reconstructing Spatio-Temporal Trajectories of Visual Object Memories in the Human Brain

Abstract

Our understanding of how information unfolds when we recall events from memory remains limited. In this study, we investigate whether the reconstruction of visual object memories follows a backward trajectory along the ventral visual stream with respect to perception, such that their neural feature representations are gradually reinstated from late areas close to the hippocampus backwards to lower-level sensory areas. We use multivariate analyses of fMRI activation patterns to map high-and low-level features of the object memories onto the brain during retrieval, and EEG-fMRI fusion to track the temporal evolution of the reactivated patterns. Participants studied new associations between verbs and randomly paired object images in an encoding phase, and subsequently recalled the objects when presented with the corresponding verb cue. Decoding reactivated memory features from fMRI activity revealed that retrieval patterns were dominated by conceptual features, represented in comparatively late visual and parietal areas. Representational-similarity-based fusion then allowed us to map the EEG patterns that emerged at each given time point of a trial onto the spatially resolved fMRI patterns. This fusion suggests that memory reconstruction proceeds from anterior fronto-temporal to posterior occipital and parietal regions, partly in line with a semantic-to-perceptual gradient. A linear regression on the peak time points of reactivated brain regions statistically confirms that the temporal progression is reversed with respect to encoding in ventral stream areas. Together, the results shed light onto the spatio-temporal trajectories along which memories and their constituent features are reconstructed during associative retrieval. Significance statementIn our work, we combined EEG and fMRI to investigate which features of a visual object are reactivated when recalled from episodic memory, and how the memory reconstruction stream unfolds over time and across the brain. Our findings suggest that with respect to perception, memory retrieval follows a backwards information trajectory along a conceptual-to-perceptual gradient, and additionally relays retrieved information to multimodal fronto-parietal brain regions. These findings address the fundamental question of whether memories are more or less complete and truthful reconstructions of past events, or instead are subject to systematic biases that prioritise some types of features over others. Our data suggests that episodic memory retrieval is a dynamic and highly reconstructive process with clear prioritisation of abstract-conceptual over detailed-perceptual information.