Biology subjects

Karapetian, A.

Publications and source records attributed to Karapetian, A..

4 recordsLinked to original sources

bioRxiv2025

Investigating the temporal dynamics and modelling of mid-level feature representations in humans

Scene perception is a key function of biological visual systems. According to the hierarchical processing view, scene perception in the human brain begins with low-level features, progresses to mid-level features, and ends with high-level features. While low- and high-level feature processing is well-studied, research on mid-level features remains limited. Here, we addressed this gap by investigating when mid-level features are processed in humans using a novel stimulus set of naturalistic scenes as images and videos, accompanied with ground-truth annotations for five mid-level features (reflectance, lighting, world normals, scene depth and skeleton position), and two framing features: one low-level (edges) and one high-level feature (action). To reveal when low-, mid- and high-level features are represented in the brain, we collected electroencephalography (EEG) data from human participants during stimulus presentation and trained encoding models to predict EEG data from ground-truth annotations. We revealed that mid-level features were best represented between [~]100 and [~]250 ms post-stimulus, between low- and high-level features. Moreover, we assessed scene- and action-trained convolutional neural networks (CNNs) as models of mid-level feature processing in humans. We found a comparable processing order for mid-but not low- or high-level features with humans. Overall, our results characterize mid-level feature processing in humans in the temporal domain and reveal CNNs as suitable models of the processing hierarchy of mid-level vision in humans.

neuroscience↗

bioRxiv2023

The link between visual representations and behavior in human scene perception

1.Scene recognition is a core sensory capacity that enables humans to adaptively interact with their environment. Despite substantial progress in the understanding of the neural representations underlying scene recognition, the relevance of these representations for behavior given varying task demands remains unknown. To address this, we aimed to identify behaviorally relevant scene representations, to characterize them in terms of their underlying visual features, and to reveal how they vary across different tasks. We recorded fMRI data while human participants viewed scenes and linked brain responses to behavior in three tasks acquired in separate sessions: manmade/natural categorization, basic-level categorization, and fixation color discrimination. We found correlations between categorization response times and scene-specific brain responses, quantified as the distance to a hyperplane derived from a multivariate classifier. Across tasks, these effects were found in largely distinct parts of the ventral visual stream. This suggests that different scene representations are relevant for behavior depending on the task. Next, using deep neural networks as a proxy for visual feature representations, we found that early/intermediate layers mediated the relationship between scene representations and behavior for both categorization tasks, indicating a contribution of low-/mid-level visual features to these representations. Finally, we observed opposite patterns of brain-behavior correlations in the manmade/natural and the fixation task, indicating interference of representations with behavior for task demands that do not align with the content of representations. Together, these results reveal the spatial extent, content, and task-dependence of the visual representations that mediate behavior in complex scenes.

neuroscience↗

bioRxiv2023

Empirically identifying and computationally modelling the brain-behaviour relationship for human scene categorization

Humans effortlessly make quick and accurate perceptual decisions about the nature of their immediate visual environment, such as the category of the scene they face. Previous research has revealed a rich set of cortical representations potentially underlying this feat. However, it remains unknown which of these representations are suitably formatted for decision-making. Here, we approached this question empirically and computationally, using neuroimaging and computational modelling. For the empirical part, we collected electroencephalography (EEG) data and reaction times from human participants during a scene categorization task (natural vs. man-made). We then related neural representations to behaviour using a multivariate extension of signal detection theory. We observed a correlation specifically between [~]100 ms and [~]200 ms after stimulus onset, suggesting that the neural scene representations in this time period are suitably formatted for decision-making. For the computational part, we evaluated a recurrent convolutional neural network (RCNN) as a model of brain and behaviour. Unifying our previous observations in an image-computable model, the RCNN predicted well the neural representations, the behavioural scene categorization data, as well as the relationship between them. Our results identify and computationally characterize the neural and behavioural correlates of scene categorization in humans. Significance statementCategorizing scene information is a ubiquitous and crucial task. Here we provide an empirical and computational account of scene categorization. Previous research has identified when scenes are represented in the visual processing hierarchy, but it remains unclear which of these representations are relevant for behaviour. We identified such representations between [~]100 ms and [~]200 ms after stimulus onset. We then showed that scene categorization in humans can be modelled via a recurrent convolutional neural network in a unified manner, i.e., in terms of neural and behavioural correlates, and their relationship. Together this reveals which representations underlie scene categorization behaviour and proposes a computational mechanism that implements such representations.

neuroscience↗

bioRxiv2021

Resolving the time course of visual and auditory object categorization

Humans can effortlessly categorize objects, both when they are conveyed through visual images and spoken words. To resolve the neural correlates of object categorization, studies have so far primarily focused on the visual modality. It is therefore still unclear how the brain extracts categorical information from auditory signals. In the current study we used EEG (N=47) and time-resolved multivariate pattern analysis to investigate (1) the time course with which object category information emerges in the auditory modality and (2) how the representational transition from individual object identification to category representation compares between the auditory modality and the visual modality. Our results show that (1) that auditory object category representations can be reliably extracted from EEG signals and (2) a similar representational transition occurs in the visual and auditory modalities, where an initial representation at the individual-object level is followed by a subsequent representation of the objects category membership. Altogether, our results suggest an analogous hierarchy of information processing across sensory channels. However, we did not find evidence for a shared supra-modal code, suggesting that the contents of the different sensory hierarchies are ultimately modality-unique.

animal behavior and cognition↗