Biology subjects

Nastase, S. A.

Publications and source records attributed to Nastase, S. A..

At least 19 recordsLinked to original sources

bioRxiv2025

Hierarchical systems in the default mode network when reasoning about self and other mental states

Humans spend time contemplating the minds of others. But this ability is not limited to external agents - we also turn the lens for reading minds inward, reflecting on our own thoughts, emotions, and sense of self. Some processes involved in reasoning about minds may rely on shared mechanisms, while others may be specific to the agent under consideration. We developed a paradigm where participants performed either a mental state inference task or a control task targeting either another person presented onscreen or their own mind. Using fMRI and multi-voxel pattern analysis, we replicate a well-established self-other axis along the medial wall of prefrontal cortex: ventral regions selectively decoded mental state inference patterns for self, but not other, whereas more dorsal regions decoded mental state inference for both self and other, compared to control conditions. Posterior cingulate cortex, on the other hand, differentiated the target of mental state inference. Using a cross-classification analysis, we also found patterns in the dorsomedial prefrontal cortex, ventromedial prefrontal cortex, and right temporoparietal junction were sensitive to mental state reasoning in general, regardless of the target agent. These findings highlight one process reflecting reasoning specific to the agent and another reflecting the reasoning process itself.

neuroscience↗

bioRxiv2025

Cortical language areas are coupled via a soft hierarchy of model-based linguistic features

Natural language comprehension is a complex task that relies on coordinated activity across a network of cortical regions. In this study, we propose that regions of the language network are coupled to one another through subspaces of shared linguistic features. To test this idea, we developed a model-based connectivity framework to quantify stimulus-driven, feature-specific functional connectivity between language areas during natural language comprehension. Using fMRI data acquired while subjects listened to spoken narratives, we tested three types of features extracted from a unified neural network model for speech and language: low-level acoustic embeddings, mid-level speech embeddings, and high-level language embeddings. Our modeling framework enabled us to quantify the proportions of stimulus features driving connectivity between regions: early auditory areas were coupled to intermediate language areas via lower-level acoustic and speech features; in contrast, higher-order language and default-mode regions were predominantly coupled through more abstract language features. We observed a clear progression of feature-specific connectivity from early auditory to lateral temporal areas, advancing from acoustic connectivity to speech- and finally to language-driven connectivity. Our findings suggest that higher-order language areas are coupled along increasingly higher level, more contextualized language features.

neuroscience↗

bioRxiv2025

Context modulates brain state dynamics and behavioral responses during narrative comprehension

Narrative comprehension is inherently context-sensitive, yet the brain and cognitive mechanisms by which brief contextual priming shapes story interpretation remain unclear. Using hidden Markov modeling (HMM) of fMRI data, we identified dynamic brain states as participants listened to an ambiguous spoken story under two distinct narrative contexts (affair vs. paranoia). We identified recurrent states involving auditory, language, and default mode network (DMN) regions that were expressed across both groups, as well as additional states characterized by recruitment of multiple-demand network (MDN) systems, including control, dorsal attention, and salience networks. Bayesian mixed-effects modeling revealed that contextual framing modulated how specific linguistic and character-related features influenced the probability of occupying these states. Complementary behavioral data showed parallel context-sensitive modulation of participants moment-to-moment interpretive judgments. Together, these findings suggest that contextual priming influences narrative comprehension through subtle, feature-dependent adjustments in the engagement of DMN- and MDN-related brain states during naturalistic story listening.

neuroscience↗

bioRxiv2025

Linguistic coupling between neural systems for speech production and comprehension during real-time dyadic conversations

The core use of human language is communicating complex ideas from one mind to another in everyday conversations. In conversations, comprehension and production processes are intertwined, as speakers soon become listeners, and listeners become speakers. Nonetheless, the neural systems underlying these faculties are typically studied in isolation, using paradigms that cannot fully engage our capacity for interactive communication, and with indirect measures of similarity. Here, we used an fMRI hyperscanning paradigm to measure neural activity simultaneously in pairs of subjects engaged in real-time, interactive conversations. We used contextual word embeddings from a large language model to quantify the linguistic coupling between production and comprehension systems within and across individual brains. We found a highly overlapping network of regions involved in both production and comprehension spanning much of the cortical language network. Our findings reveal that shared representations for both processes extend beyond the language network into areas associated with social cognition. Together, these results suggest that the specialized neural systems for speech perception and production align on a common set of linguistic features encoded in a broad cortical network for language and communication.

neuroscience↗

bioRxiv2025

The "Podcast" ECoG dataset for modeling neural activity during natural language comprehension

Naturalistic electrocorticography (ECoG) data are a rare but essential resource for studying the brains linguistic capabilities. ECoG offers a high temporal resolution suitable for investigating processes at multiple temporal timescales and frequency bands. It also provides broad spatial coverage, often along critical language areas. Here, we share a dataset of nine ECoG participants with 1,330 electrodes listening to a 30-minute audio podcast. The richness of this naturalistic stimulus can be used for various research endeavors, from auditory perception to semantic integration. In addition to the neural data, we extract linguistic features of the stimulus ranging from phonetic information to large language model word embeddings. We use these linguistic features in encoding models that relate stimulus properties to neural activity. Finally, we provide detailed tutorials for preprocessing raw data, extracting stimulus features, and running encoding analyses that can serve as a pedagogical resource or a springboard for new research.

neuroscience↗

bioRxiv2025

Action features dominate cortical representation during natural vision

Cortical resources are allocated into systems specialized for processing ecologically relevant features of the world. These features are typically studied in isolation using controlled experimental stimuli, making it difficult to assess the relative importance of different kinds of features that tend to overlap during natural vision. In the current study, we evaluated the relative contributions of action, agent, and scene features in predicting cortical activity while participants viewed a 1-hour nature documentary. We tested four sets of model features: semantic vectors capturing the observed action, agent, and scene features derived from an annotation of the stimulus, as well as low-level visual motion-energy features. We used banded ridge regression to fit vertex-wise encoding models from the combination of all four sets of features. While each set of features predicted neural activity in the expected areas, the action features predicted more widespread activity. A variance partitioning analysis revealed that the action features captured the most unique variance in neural activity across ten times as many cortical vertices as the agent or scene features. Our findings suggest that cortical activity during dynamic, natural vision is dominated by features for understanding the actions of others.

neuroscience↗

bioRxiv2024

Behaviorally-relevant features of observed actions dominate cortical representational geometry in natural vision

We effortlessly extract behaviorally relevant information from dynamic visual input in order to understand the actions of others. In the current study, we develop and test different classes of models to better understand the neural representational geometries supporting action understanding. Using fMRI, we measured brain activity as participants viewed a diverse set of 90 different video clips depicting social and nonsocial actions in real-world contexts. We developed five behavioral models using arrangement tasks: two models reflecting behavioral judgments of the purpose (transitivity) and the social content (sociality) of the actions depicted in the video stimuli; and three models reflecting behavioral judgments of the visual content (people, objects, and scene) depicted in still frames of the stimuli. We evaluated how well these models predict neural representational geometry and tested them against semantic models based on verb and nonverb embeddings and visual models based on gaze and motion energy. Our results revealed that behavioral models of action meaning/purpose (transitivity, sociality) better reflect neural representational geometry than behavioral models of object, person, and scene features, as well as semantic and visual models throughout much of cortex. The sociality and transitivity models captured a large portion of unique variance throughout the action observation network, extending into regions not typically associated with action perception, like ventral temporal cortex. Overall, these findings expand our understanding of the action observation network and indicate that the social content and purpose of observed actions are predominant in cortical representation.

neuroscience↗

bioRxiv2024

Information-making processes in the speaker's brain drive human conversations forward

Natural human conversation is driven by the exchange of information-rich messages that surprise the listener and deviate from predictable context. While extensive research has characterized how the brain processes unexpected linguistic input during comprehension, the neural mechanisms underlying the generation of such information by the speaker remain poorly understood. In this study, we hypothesize that the speakers brain actively creates information through extra neural computation, transforming internal thoughts into novel, meaningful linguistic outputs. Utilizing a unique 24/7 dataset of continuous electrocorticography (ECoG) comprising approximately 100 hours of spontaneous natural conversations, we contrast the neural basis of speech production and comprehension within the same participants. Behaviorally, we find that speakers take longer pausing for an additional 100-150 milliseconds before producing information-rich, improbable words, even when controlling for word identity and frequency. Neurally, we provide converging evidence for a previously unreported process: generating information-rich words elicits significantly stronger neural activity (ERP) and enhanced neural encoding and decoding in language-related areas starting 100-500 ms before articulation. This pattern contrasts sharply with speech comprehension, where enhanced neural activity for predictable words occurs before onset, while responses to improbable words emerge only after onset as prediction errors. Furthermore, we demonstrate that large language models (LLMs) mirror this biological process, requiring deeper internal computation across layers to generate improbable versus probable words. Together, these results reveal that the speakers brain is not merely a transmission channel but an active generator of information, recruiting extra neural resources to create novel content that diverges from listener expectations.

neuroscience↗

bioRxiv2024

ALIGNING BRAINS INTO A SHARED SPACE IMPROVES THEIR ALIGNMENT TO LARGE LANGUAGE MODELS

AO_SCPLOWBSTRACTC_SCPLOWRecent studies have shown that large language models (LLMs) can accurately predict neural activity measured using electrocorticography (ECoG) during natural language processing. To predict word-by-word neural activity, most prior work has estimated and evaluated encoding models within each electrode and subject--without evaluating how these models generalize across individual brains. In this paper, we analyze neural responses in 8 subjects while they listened to the same 30-minute podcast episode. We use a shared response model (SRM) to estimate a shared information space across subjects. We show that SRM significantly improves LLM-based encoding model performance. We also show that we can use this shared space to denoise the individual brain responses by projecting back into the individualized electrode space, and this process achieves a mean 38% improvement in encoding performance. The strongest improvement was observed for brain areas specialized for language comprehension, specifically in the superior temporal gyrus (STG) and inferior frontal gyrus (IFG). Critically, estimating a shared space allows us to construct encoding models that better generalize across individuals.

neuroscience↗

bioRxiv2023

How a speaker herds the audience: Multi-brain neural convergence over time during naturalistic storytelling

Storytelling--an ancient way for humans to share individual experiences with others--has been found to induce neural synchronization among listeners. In our exploration of the dynamic fluctuations in listener-listener (LL) coupling throughout stories, we uncover a significant correlation between LL and lag-speaker-listener (lag-SL) couplings over time. Using the analogy of neural pattern (dis)similarity as distances between participants, we term this phenomenon the "herding effect": like a shepherd guiding a group of sheep, the more closely listeners follow the speakers prior brain activity patterns (higher lag-SL similarity), the more tightly they cluster together (higher LL similarity). This herding effect is particularly pronounced in brain regions where neural synchronization among listeners tracks with behavioral ratings of narrative engagement, highlighting the mediating role of narrative content in the observed multi-brain neural coupling dynamics. By integrating LL and SL neural couplings, this study illustrates how unfolding stories shape a dynamic multi-brain functional network and how the configuration of this network may be associated with moment-by-moment efficacy of communication. Significance StatementDifferent stories have been found to evoke distinct brain activation patterns in the audience. This study delves into how the storyteller guides the audience through the multi-dimensional space of brain states, reflected in a series of shared activation patterns. We reveal that the listeners follow along the trajectory outlined by the speakers brain activity moments before, forming a tighter cluster at the more engaging moments of the story. This phenomenon is localized to high-level cortical areas supporting event representation. Our investigation illustrates how storytelling dynamically sculpts multi-brain neural dynamics in both the listeners and the speaker, shedding light on the potential association between the configuration of this network and communication efficacy.

neuroscience↗

bioRxiv2023

A shared linguistic space for transmitting our thoughts from brain to brain in natural conversations

Effective communication hinges on a mutual understanding of word meaning in different contexts. The embedding space learned by large language models can serve as an explicit model of the shared, context-rich meaning space humans use to communicate their thoughts. We recorded brain activity using electrocorticography during spontaneous, face-to-face conversations in five pairs of epilepsy patients. We demonstrate that the linguistic embedding space can capture the linguistic content of word-by-word neural alignment between speaker and listener. Linguistic content emerged in the speakers brain before word articulation, and the same linguistic content rapidly reemerged in the listeners brain after word articulation. These findings establish a computational framework to study how human brains transmit their thoughts to one another in real-world contexts.

neuroscience↗

bioRxiv2023

Deep speech-to-text models capture the neural basis of spontaneous speech in everyday conversations

Humans effortlessly use the continuous acoustics of speech to communicate rich linguistic meaning during everyday conversations. In this study, we leverage 100 hours (half a million words) of spontaneous open-ended conversations and concurrent high-quality neural activity recorded using electrocorticography (ECoG) to decipher the neural basis of real-world speech production and comprehension. Employing a deep multimodal speech-to-text model named Whisper, we develop encoding models capable of accurately predicting neural responses to both acoustic and semantic aspects of speech. Our encoding models achieved high accuracy in predicting neural responses in hundreds of thousands of words across many hours of left-out recordings. We uncover a distributed cortical hierarchy for speech and language processing, with sensory and motor regions encoding acoustic features of speech and higher-level language areas encoding syntactic and semantic information. Many electrodes--including those in both perceptual and motor areas--display mixed selectivity for both speech and linguistic features. Notably, our encoding model reveals a temporal progression from language-to-speech encoding before word onset during speech production and from speech-to-language encoding following word articulation during speech comprehension. This study offers a comprehensive account of the unfolding neural responses during fully natural, unbounded daily conversations. By leveraging a multimodal deep speech recognition model, we highlight the power of deep learning for unraveling the neural mechanisms of language processing in real-world contexts.

neuroscience↗

bioRxiv2022

Cross-movie prediction of individualized functional topography

Participant-specific, functionally-defined brain areas are usually mapped with functional localizers and estimated by making contrasts between responses to single categories of input. Naturalistic stimuli engage multiple brain systems in parallel, provide more ecologically plausible estimates of real-world statistics, and are friendly to special populations. The current study shows that cortical functional topographies in individual participants can be estimated with high fidelity from naturalistic stimuli. Importantly, we demonstrate that robust, individualized estimates can be obtained even when participants watched different movies, were scanned with different parameters/scanners, and were sampled from different institutes across the world. Our results create a foundation for future studies that allow researchers to estimate a broad range of functional topographies based on naturalistic movies and a normative database, making it possible to integrate high-level cognitive functions across datasets from laboratories worldwide.

neuroscience↗

bioRxiv2022

Correspondence between the layered structure of deep language models and temporal structure of natural language processing in the human brain

Deep language models (DLMs) provide a novel computational paradigm for how the brain processes natural language. Unlike symbolic, rule-based models described in psycholinguistics, DLMs encode words and their context as continuous numerical vectors. These "embeddings" are constructed by a sequence of computations organized in "layers" to ultimately capture surprisingly sophisticated representations of linguistic structures. How does this layered hierarchy map onto the human brain during natural language comprehension? In this study, we used electrocorticography (ECoG) to record neural activity in language areas along the superior temporal gyrus and inferior frontal gyrus while human participants listened to a 30-minute spoken narrative. We supplied this same narrative to a high-performing DLM (GPT2-XL) and extracted the contextual embeddings for each word in the story across all 48 layers of the model. We next trained a set of linear encoding models to predict the temporally-evolving neural activity from the embeddings at each layer. We found a striking correspondence between the layer-by-layer sequence of embeddings from GPT2-XL and the temporal sequence of neural activity in language areas. In addition, we found evidence for the gradual accumulation of recurrent information along the linguistic processing hierarchy. However, we also noticed additional neural processes in the brain, but not in DLMs, during the processing of surprising (unpredictable) words. These findings point to a connection between human language processing and DLMs where the layer-by-layer accumulation of contextual information in DLM embeddings matches the temporal dynamics of neural activity in high-order language areas.

neuroscience↗

bioRxiv2022

Reconstructing the cascade of language processing in the brain using the internal computations of a transformer-based language model

Humans use complex linguistic structures to transmit ideas to one another. The brain is thought to deploy specialized computations to process these structures. Recently, a new class of artificial neural networks based on the Transformer architecture has revolutionized the field of language modeling, attracting attention from neuroscientists seeking to understand the neurobiology of language in silico. Transformers integrate information across words via multiple layers of structured circuit computations, forming increasingly contextualized representations of linguistic content. Prior work has focused on the internal representations (the "embeddings") generated by these circuits. In this paper, we instead analyze the circuit computations directly: we deconstruct these computations into functionally-specialized "transformations" to provide a complementary window onto linguistic computations in the human brain. Using functional MRI data acquired while participants listened to naturalistic spoken stories, we first verify that the transformations account for considerable variance in brain activity across the cortical language network. We then demonstrate that the emergent syntactic computations performed by individual, functionally-specialized "attention heads" differentially predict brain activity in specific cortical regions. These heads fall along gradients corresponding to different layers, contextual distances, and syntactic dependencies in a low-dimensional cortical space. Our findings indicate that large language models and the cortical language network may converge on similar trends of functional specialization for processing natural language.

neuroscience↗

bioRxiv2022

The Individualized Neural Tuning Model: Precise and generalizable cartography of functional architecture in individual brains

Quantifying how brain functional architecture differs from person to person is a key challenge in human neuroscience. Current individualized models of brain functional organization are based on brain regions and networks, limiting their use in studying fine-grained vertex-level differences. In this work, we present the Individualized Neural Tuning (INT) model, a fine-grained individualized model of brain functional organization. The INT model is designed to have vertex-level granularity, to capture both representational and topographic differences, and to model stimulus-general neural tuning. Through a series of analyses, we demonstrate that (a) our INT model provides a reliable individualized measure of fine-grained brain functional organization, (b) it accurately predicts individualized brain response patterns to new stimuli, and (c) it requires only 10-20 minutes of data for good performance. The high reliability, specificity, precision, and generalizability of our INT model affords new opportunities for building brain-based biomarkers based on naturalistic neuroimaging paradigms.

neuroscience↗

bioRxiv2022

Brain embeddings with shared geometry to artificial contextual embeddings, as a code for representing language in the human brain

Contextual embeddings, derived from deep language models (DLMs), provide a continuous vectorial representation of language. This embedding space differs fundamentally from the symbolic representations posited by traditional psycholinguistics. Do language areas in the human brain, similar to DLMs, rely on a continuous embedding space to represent language? To test this hypothesis, we densely recorded the neural activity in the Inferior Frontal Gyrus (IFG, also known as Brocas area) of three participants using dense intracranial arrays while they listened to a 30-minute podcast. From these fine-grained spatiotemporal neural recordings, we derived for each patient a continuous vectorial representation for each word (i.e., a brain embedding). Using stringent, zero-shot mapping, we demonstrated that brain embeddings in the IFG and the DLM contextual embedding space have strikingly similar geometry. This shared geometry allows us to precisely triangulate the position of unseen words in both the brain embedding space (zero-shot encoding) and the DLM contextual embedding space (zero-shot decoding). The continuous brain embedding space provides an alternative computational framework for how natural language is represented in cortical language areas.

neuroscience↗

bioRxiv2022

Real-time neurofeedback to alter interpretations of a naturalistic narrative

We explored the potential of using real-time fMRI (rt-fMRI) neurofeedback training to bias interpretations of naturalistic narrative stimuli. Participants were randomly assigned to one of two possible conditions, each corresponding to a different interpretation of an ambiguous spoken story. While participants listened to the story in the scanner, neurofeedback was used to reward neural activity corresponding to the assigned interpretation. After scanning, final interpretations were assessed. While neurofeedback did not change story interpretations on average, participants with higher levels of decoding accuracy during the neurofeedback procedure were more likely to adopt the assigned interpretation. Thus, we believe that individualized neurofeedback shaped interpretations successfully when the signal was accurate, although more work is needed to improve this method and validate the result. While naturalistic stimuli introduce a unique set of challenges in providing effective and individualized neurofeedback, we believe that this technique holds promise for individualized cognitive therapy.

neuroscience↗