Triple-N Dataset: Non-human Primate Neural Responses to Natural Scenes

Abstract

Understanding the neural mechanisms of visual perception requires data that encompass both large-scale cortical activity and the fine-grained dynamics of individual neurons. While the Natural Scenes Dataset (NSD) has provided substantial insights into visual processing in humans (Allen et al., 2022), its reliance on functional magnetic resonance imaging (fMRI) limits the exploration of individual neuron contributions. To bridge this gap, we present a new dataset: the Triple-N (Non-human Primate Neural Responses to Natural Scenes) dataset, which extends the NSD framework to non-human primates, incorporating single-neuron activity and fMRI recorded from the inferotemporal (IT) cortex. Using Neuropixels probes, we recorded neural responses from five macaques as they passively viewed 1,000 shared NSD images. The dataset includes 59 sessions across 27 sub-regions, capturing over 30,000 visual responsive units. Many recordings were obtained from fMRI-defined category-selective regions, such as face-, body-, scene-, and color-selective areas. Our dataset enables in-depth exploration of neural responses at multiple levels - from population dynamics to single-neuron activity, providing new insights into various aspects of visual processing, including the heterogeneity of object selectivity within functional regions and the temporal dynamics of responses to natural images. Furthermore, the dataset enables joint cross-species analyses, by integrating the macaque neural recordings with human fMRI data, offering a framework for comparing and aligning visual representations across primate species. Overall, our dataset provides a valuable resource for advancing our understanding of visual perception, bridging the gap between large-scale neuroimaging and fine-grained electrophysiological signals, while also facilitating the development of computational models of the high-level visual system.