Biology subjects

Yovel, G.

Publications and source records attributed to Yovel, G..

14 recordsLinked to original sources

bioRxiv2026

Visual and auditory deep learning models capture neural representations of naturalistic social interaction in the superior temporal sulcus

The superior temporal sulcus (STS) is selectively responsive to multimodal social interactions. Yet studies so far have relied on pre-defined, simplified stimuli or features to uncover the type of information that drives STS activity. We hypothesized that high-dimensional representations from visual and auditory deep learning models would better predict STS responses to naturalistic social interactions. We used self-supervised visual and auditory deep learning models to extract representations of movie frames and audio, respectively, of a TV series participants watched during fMRI scanning. Voxel-wise encoding models of a joint visual-auditory representation outperformed human-made social-affective annotations in predicting STS. Variance partition further revealed visual-auditory posterior-to-anterior gradient within the STS. To interpret what these models encode, we applied Principal Component Analysis to the encoding model weights. In both the visual and auditory models the first dimension tracked social interaction and peaked in the STS, indicating that social interaction is a dominant dimension of STS representation across both modalities. We conclude that the STS represents naturalistic social interaction in a multimodal manner, integrating visual and auditory information, and that visual and auditory deep learning models capture key representational properties of these responses.

neuroscience↗

bioRxiv2026

Altering facial preferences with context

Facial features are known to play a key role in how we judge a persons traits. However, in many scenarios people do not evaluate a person in isolation but are required to compare and choose between multiple candidates. Yet, the principles that govern these decisions remain unknown. In economic literature, decisions are known to be influenced by the context of available options, but it is unclear to what extent these principles apply to social decisions about naturalistic stimuli such as human faces. Here, we bridged between these two fields of research - face evaluations and economic decisions - to investigate whether facial preferences can be systematically shifted by context, exhibiting a well-known choice context effect, the "decoy effect". We asked participants to choose the person they think is more trustworthy from either a set of two ("target" and "competitor") or three faces (including a "decoy"). Based on economic theory, we carefully selected face sets using their trustworthiness rankings and perceptual similarity, and tested them across several exploratory samples (n=100-400) and one large, preregistered experiment (n=1099). We found that adding a decoy face to the choice set increased participants choices of the target option by up to 7%, while the decoy face was rarely selected, consistent with a typical decoy effect. These results provide direct evidence that social judgements are affected by the same context-dependent principles known from economic choices, and that the decoy effect can be generalized to complex naturalistic stimuli such as human faces, relevant to various real-world applications. Significance statementPeople often rely on facial judgements when choosing between candidates, like politicians or potential partners. These judgments are typically made in comparison to available alternatives rather than in isolation. In economic literature, decisions are known to depend on the context of alternatives, but it remains unclear whether social evaluations follow the same principles. Here, tested whether the "decoy effect", a well-known choice context effect, applies to social judgments. Participants were asked to choose the more trustworthy individual from either two (decoy-absent) or three faces (decoy-present). Adding a decoy face increased selection of the target face by up to 7%, while the decoy itself was rarely chosen, showing that social judgments are governed by the same context-dependent principles as economic choices.

neuroscience↗

bioRxiv2026

Enterovirus D68 VP1 and VP3 determine neurotropism in human spinal cord organoids

Enterovirus D68 (EV-D68) is a non-polio enterovirus that can cause a polio-like paralysis condition, acute flaccid myelitis (AFM). EV-D68 associated AFM cases waned in the US after 2018 and the reasons for this are unknown. It has recently been demonstrated that EV-D68 containing point mutations in viral structural proteins VP1 and VP3 resulted in decreased paralysis in different neonatal mouse models. However, phenotypes of these mutations in a human multicellular central nervous system (CNS) model are unknown. We hypothesized that mutations in VP1 and VP3 will similarly direct neurotropism in human spinal cord organoids (hSCO). To investigate this, we recreated viruses with mutations in VP3 (I88V) or VP1 (L1I/N2D/T98A/E283K or L1P/V148A/K282R) and infected hSCOs. We found that VP3 I88V and VP1 L1I/N2D/T98A/E283K resulted in decreased titer and viral protein staining, consistent with attenuated neurovirulence in previously published murine models. When these mutations were combined, their effects on neurotropism were not additive. Sequence analysis of recently circulating EV-D68 strains reveals that VP3 I88 and VP1 E283 have remained the dominant amino acid residues since 2014, whereas VP1 sites 1, 2, and 98 have higher population diversity, indicating that these residues may be contributing to newly reduced neurovirulence after 2018.

microbiology↗

bioRxiv2025

A developmentally inspired computational model of face recognition that learns continuously through generative memory replay

Deep convolutional neural networks (DCNNs) have emerged as powerful models for human face recognition, capturing several hallmark behavioral phenomena such as the face inversion effect and the other-race effect. Yet, key differences remain between how DCNNs and humans, particularly infants, learn to recognize faces. In this study, we present a developmentally-inspired model that addresses three critical gaps between humans and standard DCNNs: (1) DCNNs are trained via one-time batch learning, whereas infants encounter faces gradually over time; (2) DCNNs are trained on thousands of face images with high variability of static images for each identity, while infants are initially exposed to a small number of identities, often limited to close caregivers, through a continuous, dynamic stream of visual input with lower within-identity variability; (3) The goal of DCNNs is to recognize untrained (unfamiliar) faces, while the goal of human face system is to recognize familiar faces. To better approximate developmental face learning, we introduce a model comprising three interacting modules: an Embedder - a DCNN trained to encode face identity continually on up to 10 identities, based on images taken from videos of a TV series; an Autoencoder used for generative memory-replay, to construct images of missing identities in each training phase; and a Memory module - stores identity-specific latent codes. Results show that this model achieves recognition performance comparable to batch-trained DCNNs. We propose this new model as a framework for studying the developmental and mature mechanisms of human face recognition.

neuroscience↗

bioRxiv2025

Self-supervision deep learning models are better models of human high-level visual cortex: The roles of multi-modality and dataset training size

With the rapid development of Artificial Neural Network based visual models, many studies have shown that these models show unprecedented potence in predicting neural responses to images in visual cortex. Lately, advances in computer vision have introduced self-supervised models, where a model is trained using supervision from natural properties of the training set. This has led to examination of their neural prediction performance, which revealed better prediction of self-supervised than supervised models for models trained with language supervision or with image-only supervision. In this work, we delve deeper into the models ability to explain neural representations of object categories. We compare models that differed in their training objectives to examine where they diverge in their ability to predict fMRI and MEG recordings while participants are presented with images of different object categories. Results from both fMRI and MEG show that self-supervision was advantageous in comparison to classification training. In addition, language supervision is a better predictor for later stages of visual perception, while image-only supervision shows a consistent advantage over a longer duration, beginning from 80ms after exposure. Examination of the effect of data size training revealed that large dataset did not necessarily improve neural predictions, in particular in visual self-supervised models. Finally, examination of the correspondence of the hierarchy of each model to visual cortex showed that image-only self-supervision led to better correspondence than image only models. We conclude that while self-supervision shows consistently better prediction of fMRI and MEG recordings, each type of supervision reveals a different property of neural activity, with language-supervision explaining later onsets, while image-only self-supervision explains long and very early latencies of the neural response, with the model hierarchy naturally sharing corresponding hierarchical structure as the brain.

neuroscience↗

bioRxiv2024

Recombinant B3 clade enterovirus D68 strains are efficiently rescued in 293T cells and infect human spinal cord organoids.

Enterovirus D68 (EV-D68) is associated with respiratory disease in children. Between 2014 and 2018, biennial EV-D68 outbreaks coincided with peaks of a polio-like neurologic condition called acute flaccid myelitis (AFM). We hypothesized that specific mutations within the currently circulating B3 clade of EV-D68 impacted neurovirulence. However, recovery of these strains from infectious clones by published methods proved unsuccessful. Therefore, we tested different cell lines, reagents, and conditions to enhance efficiency of recombinant (r)EV-D68 rescue. In this study, we present a tractable rescue system to define virulence determinants in B3 clade strains. Using this approach, we successfully rescued historic and contemporary rEV-D68 strains to high titer after limited cell culture passage. All strains in our study replicated efficiently in the parental RD cell line and in human respiratory epithelial cell lines, with BEAS-2B cells exhibiting greater permissivity than A549 cells. While B2 and B3 clade strains could infect the SH-SY5Y neuroblastoma cell line, the neurovirulent B2 clade strain rUSA/IL/2014-18952 was more dependent on neuron differentiation than B3 clade strains and replication was not sustained under multi-cycle growth conditions. Conversely, replication of rUSA/IL/2014-18952 was more efficient in human spinal cord organoids, which model the cellular heterogeneity of the spinal cord, while replication of B3 clade strains was more modest. Immunofluorescence staining confirmed infection, with viral antigen colocalized with neurons. These findings suggest shifting dynamics of EV-D68 neurovirulence and provide a critical platform for further assessment of viral determinants of neurovirulence in B3 clade strains. IMPORTANCEEnterovirus D68 (EV-D68) can cause acute flaccid myelitis (AFM), a debilitating neurological condition of the spinal cord in children. Identifying viral determinants of EV- D68 neuropathogenesis is critical to understanding recent shifts in AFM prevalence; however, these investigations are limited to a small subset of infectious clones distantly related to currently circulating B3 clade strains. In this study, we leverage improved rescue strategies to characterize recombinant (r)EV-D68 strains from the dominant B3 clade. While all rEV-D68 replicate efficiently in the parental cell line and in human respiratory epithelial cells, B3 clade strains achieved greater titers in cultured neurons than a neurovirulent B2 clade strain, with less dependence on neuronal differentiation state. All B3 clade strains established infection in human spinal cord organoids, but replication varied between strains. Therefore, our study presents a tractable rescue system to begin to dissect viral determinants of shifting neurotropism within contemporary EV-D68 clades.

microbiology↗

bioRxiv2024

The organization of high-level visual cortex is aligned with visual rather than abstract linguistic information

A fundamental question dominating the study of human visual cortex is whether it is organized along visual or semantic information. This question is unresolved, and the controversy has been rekindled by the recent report that surprisingly, revealed that the representations of textual description of images by linguistic artificial networks successfully predict the response of high-level visual cortex to visual images. These findings appear to support a linguistic, abstract, organizing principle of human visual cortex. Here, using iEEG recordings from high level visual cortex in patients, we contributed to this debate, by testing the hypothesis that this linguistic alignment is restricted to textual descriptions of the visual content of the images (visual text) and does not extend to abstract textual descriptions (abstract text). We selected images that depict familiar faces and places, as these images allow for the best dissociation between these two types of text and generated their visual and abstract (e.g., name and biography of a person) textual descriptions. We then predicted the relational structures of the iEEG response to the images using their textual representations based on a large language model and the image representation based on a convolutional neural network. Neural relational-structures in high-level visual cortex were similarly predicted by images and visual-text but not abstract-text representations. Abstract text best predicted responses of the fronto-parietal cortex to the images. These results demonstrate that visual-language alignment in high-level visual cortex is limited to visually grounded language.

neuroscience↗

bioRxiv2024

Concurrent emergence of view invariance, sensitivity to critical features, and identity face classification through visual experience: Insights from deep learning algorithms

Visual experience is known to play a critical role in face recognition. This experience is believed to enable the formation of a view-invariant representation, by learning which features are critical for face identification across views. Discovering these critical features and the type of experience that is needed to uncover them is challenging. We have recently revealed a subset of facial features that are critical for human face recognition. We further revealed that deep convolutional neural networks (DCNNs) that are trained on face classification, but not on object categorization, are sensitive to these facial features, highlighting the importance of experience with faces for the system to reveal these critical features. These findings enable us now to ask what type of experience with faces is required for the network to become sensitive to these human-like critical features and whether it is associated with the formation of a view-invariant representation and face classification performance. To that end, we systematically manipulated the number of within-identity and between-identity face images and examined its effect on the network performance on face classification, view-invariant representation, and sensitivity to human-like critical facial features. Results show that increasing the number of images per identity as well as the number of identities were both required for the simultaneous development of a view-invariant representation, sensitivity to human-like critical features, and successful identity classification. The concurrent emergence of sensitivity to critical features, view invariance and classification performance through experience implies that they depend on similar features. Overall, we show how systematic manipulation of the training diet of DCNNs can shed light on the role of experience on the generation of human-like representations.

neuroscience↗

bioRxiv2024

Text-related functionality of visual human pre-frontal activations revealed through neural network convergence

The functional role of visual activations of human pre-frontal cortex remains a deeply debated question. Its significance extends to fundamental issues of functional localization and global theories of consciousness. Here we addressed this question by comparing, dynamically, the potential parallels between the relational structure of prefrontal visual activations and visual and textual-trained deep neural networks (DNNs). The frontal visual relational structures were revealed in intra-cranial recordings of human patients, conducted for clinical purposes, while the patients viewed familiar images of faces and places. Our results reveal that visual relational structures in frontal cortex were, surprisingly, predicted by text and not visual DNNs. Importantly, the temporal dynamics of these correlations showed striking differences, with a rapid decline over time for the visual component, but persistent dynamics including a significant image offset response for the text component. The results point to a dynamic text-related function of visual prefrontal responses in the human brain.

neuroscience↗

bioRxiv2022

Deep learning models of perceptual expertise support a domain-specific account

Perceptual expertise is an acquired skill that enables fine discrimination of members of a homogenous category. The question of whether perceptual expertise is mediated by general-expert or domain-specific processing mechanisms has been hotly debated for decades in human behavioral and neuroimaging studies. To decide between these two hypotheses, most studies examined whether expertise for different domains is mediated by the same mechanisms used for faces, for which most humans are expert. Here we used deep convolutional neural networks (DCNNs) to test whether perceptual expertise is best achieved by computations that are optimized for face or object classification. We re-trained a face-trained and an object-trained DCNNs to classify birds at the sub-ordinate or individual-level of categorization. The face-trained DCNN required deeper retraining to achieve the same level of performance for bird classification as an object-trained DCNN. These findings indicate that classification at the subordinate- or individual-level of categorization does not transfer well between domains. Thus, fine-grained classification is best achieved by using domain-specific rather than domain-general computations.

neuroscience↗

bioRxiv2022

Deep learning algorithms reveal a new visual-semantic representation of familiar faces in human perception and memory

Recent studies show significant similarities between the representations humans and deep neural networks (DNNs) generate for faces. However, two critical aspects of human face recognition are overlooked by these networks. First, human face recognition is mostly concerned with familiar faces, which are encoded by visual and semantic information, while current DNNs solely rely on visual information. Second, humans represent familiar faces in memory, but representational similarities with DNNs were only investigated for human perception. To address this gap, we combined visual (VGG-16), visual-semantic (CLIP), and natural language processing (NLP) DNNs to predict human representations of familiar faces in perception and memory. The visual-semantic network substantially improved predictions beyond the visual network, revealing a new visual-semantic representation in human perception and memory. The NLP network further improved predictions of human representations in memory. Thus, a complete account of human face recognition should go beyond vision and incorporate visual-semantic, and semantic representations.

neuroscience↗

bioRxiv2020

Face recognition depends on specialized mechanisms tuned to view-invariant facial features: Insights from deep neural networks optimized for face or object recognition

Faces are processed by specialized neural mechanisms in high-level visual cortex. How does this divergence to a face-specific and an object-general system contribute to face recognition? Recent advances in machine face recognition together with our understanding of how humans recognize faces enable us to address this fundamental question. We hypothesized that face recognition depends on a face-selective system that is tuned to view-invariant facial features, which cannot be accomplished by an object-general system. To test this hypothesis, we used deep convolutional neural networks (DCNNs) that were optimized for face or object recognition. Consistent with the hierarchical architecture of the visual system, results show that a human-like, view-invariant face representation emerges only at higher layers of a face-trained but not the object-trained neural network. Thus, by combining human psychophysics and computational modelling of the visual system, we revealed how the functional organization of the visual cortex may contribute to recognition.

neuroscience↗

bioRxiv2019

The functional organization of high-level visual cortex determines the representation of complex visual stimuli

A hallmark of high-level visual cortex is its functional organization of neighboring clusters of neurons that are selective to single categories such as faces, bodies and objects. However, visual scenes are typically composed of multiple categories. How does category-selective cortex represent such complex stimuli? According to a normalization mechanism, the response of a single neuron to multiple stimuli is normalized by the response of its neighboring neurons (normalization pool). Here we show that category-selectivity, measured with fMRI, can provide an estimate for the heterogeneity of the normalization pool, which determines the response to multiple stimuli. These results provide a general framework for the varying representations of multiple stimuli that were reported in different regions of category-selective cortex in neuroimaging and single-unit recording studies. This type of organization may enable a dynamic and flexible representation of complex visual scenes that can be modulated by higher-level cognitive systems according to task demands.

neuroscience↗

bioRxiv2017

The processing of dynamic faces in the human brain: Support for an integrated neural framework of face processing.

Faces convey rich information including identity, gender and expression. Current neural models of face processing suggest a dissociation between the processing of invariant facial aspects such as identity and gender, that engage the fusiform face area (FFA) and the processing of changeable aspects, such as expression and eye gaze, that engage the posterior superior temporal sulcus face area (pSTS-FA). Recent studies report a second dissociation within this network such that the pSTS-FA, but not the FFA, shows much stronger response to dynamic than static faces. The aim of the current study was to test a unified model that accounts for these two major functional characteristics of the neural face network. In an fMRI experiment, we presented static and dynamic faces while subjects judged an invariant (gender) or a changeable facial aspect (expression). We found that the pSTS-FA was more engaged in processing dynamic than static faces and changeable than invariant facial aspects, whereas the OFA and FFA showed similar response across all four conditions. Our results reveal no dissociation between the processing of changeable and invariant facial aspects, but higher sensitivity to the processing of changeable facial aspects by the motion-sensitive face area in the superior temporal sulcus.

neuroscience↗