Search bioRxiv⌕ Search

Biology subjects

Cichy, R.

Publications and source records attributed to Cichy, R..

7 recordsLinked to original sources

Self-supervised learning yields representational signatures of category-selective cortex

1.The ventral visual stream contains category-selective regions with distinct feature tuning, most prominently the fusiform face area (FFA) and parahippocampal place area (PPA). Why do these brain regions exhibit distinct tuning properties? Recent work suggests that brain-like category-selective features emerge from a general visual learning mechanism without domain-specific biases. Here, we test this proposal by applying a functional localizer approach to both humans and self-supervised neural networks, identifying face- and scene-selective units in the brain and in models. We then compared fMRI and model responses across a broad stimulus set probing classic representational signatures of the FFA and PPA, including preferences relating to curvature, animacy, real-world size, mid-level features, face shapes, and spatial layout information. Category-selective model units largely recapitulate the distinct representational signatures of category-selective brain regions, capturing most of the effects in our test battery. Our findings demonstrate that domain-general learning objectives are sufficient to create humanlike category-selectivity, suggesting that the distinct representational signatures of category-selective cortex may emerge from a unified computational goal akin to self-supervised learning.

neuroscience↗

Representational shifts from feedforward to feedback rhythms index phenomenological integration in natural vision

How does the brain integrate complex and dynamic visual inputs into phenomenologically seamless percepts? Previous results demonstrate that when visual inputs are organized coherently across space and time, they are more strongly encoded in feedback-related alpha rhythms, and less strongly in feedforward-related gamma rhythms. Here, we tested whether this representational shift from feedforward to feedback rhythms is linked to the phenomenological experience of coherence. In an EEG study, we manipulated the degree of spatiotemporal coherence by presenting two segments from the same video across visual hemifields, either synchronously or asynchronously (with a delay between segments). We asked participants whether they perceived the stimulus as coherent or incoherent. When stimuli were presented at the perceptual threshold (i.e., when the same stimulus was judged as coherent 50% of times), perception co-varied with stimulus coding across alpha and gamma rhythms: When stimuli were perceived as coherent, they were represented in alpha activity; when stimuli were perceived as incoherent, they were represented in gamma activity. Whether the same visual input is perceived as coherent or incoherent thus depends on representational shifts between feedback-related alpha and feedforward-related gamma rhythms.

neuroscience↗

Visual features are processed before navigational affordances in the human brain

To navigate through their immediate environment humans process scene information rapidly. How does the cascade of neural processing elicited by scene viewing to facilitate navigational planning unfold over time? To investigate, we recorded human brain responses to visual scenes with electroencephalography (EEG) and related those to computational models that operationalize three aspects of scene processing (2D, 3D, and semantic information), as well as to a behavioral model capturing navigational affordances. We found a temporal processing hierarchy: navigational affordance is processed later than the other scene features (2D, 3D, and semantic) investigated. This reveals the temporal order with which the human brain computes complex scene information and suggests that the brain leverages these pieces of information to plan navigation.

neuroscience↗

BOLD Moments: modeling short visual events through a video fMRI dataset and metadata

Studying the neural basis of human dynamic visual perception requires extensive experimental data to evaluate the large swathes of functionally diverse brain neural networks driven by perceiving visual events. Here, we introduce the BOLD Moments Dataset (BMD), a repository of whole-brain fMRI responses to over 1,000 short (3s) naturalistic video clips of visual events across ten human subjects. We use the videos extensive metadata to show how the brain represents word- and sentence-level descriptions of visual events and identify correlates of video memorability scores extending into the parietal cortex. Furthermore, we reveal a match in hierarchical processing between cortical regions of interest and video-computable deep neural networks, and we showcase that BMD successfully captures temporal dynamics of visual events at second resolution. With its rich metadata, BMD offers new perspectives and accelerates research on the human brain basis of visual event perception.

neuroscience↗

Time-resolved multivariate pattern analysis of infant EEG data

Time-resolved multivariate pattern analysis (MVPA), a popular technique for analyzing magneto- and electro-encephalography (M/EEG) neuroimaging data, quantifies the extent and time-course by which neural representations support the discrimination of relevant stimuli dimensions. As EEG is widely used for infant neuroimaging, time-resolved MVPA of infant EEG data is a particularly promising tool for infant cognitive neuroscience. MVPA methods have recently been applied to common infant imaging methods such as EEG and fNIRS. In this tutorial, we provide and describe code to implement time-resolved, within-subject MVPA with infant EEG data. A pipeline for time-resolved MVPA based on linear SVM classification is described and implemented with accompanying code in both Matlab and Python. Results from a test dataset indicated that in both infants and adults this method reliably produced above chance classification accuracy. Extensions of the core pipeline are presented including both geometric- and accuracy-based representational similarity analysis, implemented in Python. Common choices of implementation are presented and discussed. As the amount of artifact-free EEG data contributed by each participant is lower in studies of infants than in studies of children and adults, we also explore and discuss the impact of varying participant-level inclusion thresholds on resulting MVPA findings in these datasets.

neuroscience↗

The neural dynamics of familiar face recognition

In real-life situations, the appearance of a persons face can vary substantially across different encounters, making face recognition a challenging task for the visual system. Recent fMRI decoding studies have suggested that face recognition is supported by identity representations located in regions of the occipito-temporal cortex. Here, we used EEG to elucidate the temporal emergence of these representations. Human participants (both sexes) viewed a set of highly variable face images of four highly familiar celebrities (two male, two female), while performing an orthogonal task. Univariate analyses of event-related EEG responses revealed a pronounced differentiation between male and female faces, but not between identities of the same sex. Using multivariate representational similarity analysis, we observed a gradual emergence of face identity representations, with an increasing degree of invariance. Face identity information emerged rapidly, starting shortly after 100ms from stimulus onset. From 400ms after onset and predominantly in the right hemisphere, identity representations showed two invariance properties: (1) they equally discriminated identities of opposite sexes and of the same sex, and (2) they were tolerant to image-based variations. These invariant representations may be a crucial prerequisite for successful face recognition in everyday situations, where the appearance of a familiar person can vary drastically.\n\nSignificance StatementRecognizing the face of a friend on the street is a task we effortlessly perform in our everyday lives. However, the necessary visual processing underlying familiar face recognition is highly complex. As the appearance of a given person varies drastically between encounters, for example across viewpoints or emotional expressions, the brain needs to extract identity information that is invariant to such changes. Using multivariate analyses of EEG data, we characterize how invariant representations of face identity emerge gradually over time. After 400ms of processing, cortical representations reliably differentiated two similar identities (e.g., two famous male actors), even across a set of highly variable images. These representations may support face recognition under challenging real-life conditions.

neuroscience↗

Ultra-Rapid Serial Visual Presentation Reveals Dynamics of Feedforward and Feedback Processes in the Ventral Visual Pathway

Human visual recognition activates a dense network of overlapping feedforward and recurrent neuronal processes, making it hard to disentangle processing in the feedforward from the feedback direction. Here, we used ultra-rapid serial visual presentation to suppress sustained activity that blurs the boundaries of processing steps, enabling us to resolve two distinct stages of processing with MEG multivariate pattern classification. The first processing stage was the rapid activation cascade of the bottom-up sweep, which terminated early as visual stimuli were presented at progressively faster rates. The second stage was the emergence of categorical information with peak latency that shifted later in time with progressively faster stimulus presentations, indexing time-consuming recurrent processing. Using MEG-fMRI fusion with representational similarity, we localized recurrent signals in early visual cortex. Together, our findings segregated an initial bottom-up sweep from subsequent feedback processing, and revealed the neural signature of increased recurrent processing demands for challenging viewing conditions.

neuroscience↗