Biology subjects

Nastase, S. A.

Publications and source records attributed to Nastase, S. A..

35 records · Page 2Linked to original sources

bioRxiv2021

Information flow across the cortical timescales hierarchy during narrative comprehension

When listening to spoken narratives, we must integrate information over multiple, concurrent timescales, building up from words to sentences to paragraphs to a coherent narrative. Recent evidence suggests that the brain relies on a chain of hierarchically organized areas with increasing temporal receptive windows to process naturalistic narratives. We hypothesized that the structure of this cortical processing hierarchy should result in an observable sequence of response lags between networks comprising the hierarchy during narrative comprehension. This study uses functional MRI to estimate the response lags between functional networks during narrative comprehension. We use inter-subject cross-correlation analysis to capture network connectivity driven by the shared stimulus. We found a fixed temporal sequence of response lags--on the scale of several seconds--starting in early auditory areas, followed by language areas, the attention network, and lastly the default mode network. This gradient is consistent across eight distinct stories but absent in data acquired during rest or using a scrambled story stimulus, supporting our hypothesis that narrative construction gives rise to inter-network lags. Finally, we build a simple computational model for the neural dynamics underlying the construction of nested narrative features. Our simulations illustrate how the gradual accumulation of information within the boundaries of nested linguistic events, accompanied by increased activity at each level of the processing hierarchy, can give rise to the observed lag gradient. Significance StatementOur findings reveal a consistent, stimulus-driven gradient of lags in connectivity along the cortical processing hierarchy--from early auditory cortex to the language network, then to the default mode network--during the comprehension of naturalistic, spoken narratives. We provide a simple computational model for the neural dynamics underlying the construction of nested narrative features, allowing us to systematically explore the conditions under which the lag gradient emerges and synthesize our results with previous findings based on simple well-controlled language stimuli. Our results illustrate the isomorphism between hierarchically structured neural dynamics and hierarchically structured, real-world narrative inputs.

neuroscience↗

bioRxiv2021

Not so fast: Limited validity of deep convolutional neural networks as in silico models for human naturalistic face processing

Deep convolutional neural networks (DCNNs) trained for face identification can rival and even exceed human-level performance. The ways in which the internal face representations in DCNNs relate to human cognitive representations and brain activity are not well understood. Nearly all previous studies focused on static face image processing with rapid display times and ignored the processing of naturalistic, dynamic information. To address this gap, we developed the largest naturalistic dynamic face stimulus set in human neuroimaging research (700+ naturalistic video clips of unfamiliar faces). We used this novel naturalistic dataset to compare representational geometries estimated from DCNNs, behavioral responses, and brain responses. We found that DCNN representational geometries were consistent across architectures, cognitive representational geometries were consistent across raters in a behavioral arrangement task, and neural representational geometries in face areas were consistent across brains. Representational geometries in late, fully-connected DCNN layers, which are optimized for individuation, were much more weakly correlated with cognitive and neural geometries than were geometries in late-intermediate layers. The late-intermediate face-DCNN layers successfully matched cognitive representational geometries, as measured with a behavioral arrangement task that primarily reflected categorical attributes, and correlated with neural representational geometries in known face-selective topographies. Our study suggests that current DCNNs successfully capture neural cognitive processes for categorical attributes of faces, but less accurately capture individuation and dynamic features.

neuroscience↗

bioRxiv2021

Here's the twist: How the brain updates naturalistic event memories as our understanding of the past changes

The brain actively reshapes our understanding of past events in light of new incoming information. In the current study, we ask how the brain supports this updating process during the encoding and recall of naturalistic stimuli. One group of participants watched a movie ("The Sixth Sense") with a cinematic "twist" at the end that dramatically changed the interpretation of previous events. Next, participants were asked to verbally recall the movie events, taking into account the new "twist" information. Most participants updated their recall to incorporate the twist. Two additional groups recalled the movie without having to update their memories during recall: one group never saw the twist; another group was exposed to the twist prior to the beginning of the movie, and thus the twist information was incorporated both during encoding and recall. We found that providing participants with information about the twist beforehand altered neural response patterns during movie-viewing in the default mode network (DMN). Moreover, presenting participants with the twist at the end of the movie changed the neural representation of the previously-encoded information during recall in a subset of DMN regions. Further evidence for this transformation was obtained by comparing the neural activation patterns during encoding and recall and correlating them with behavioral signatures of memory updating. Our results demonstrate that neural representations of past events encoded in the DMN are dynamically integrated with new information that reshapes our understanding in natural contexts.

neuroscience↗

bioRxiv2021

Brain kernel: a new spatial covariance function for fMRI data

A key problem in functional magnetic resonance imaging (fMRI) is to estimate spatial activity patterns from noisy high-dimensional signals. Spatial smoothing provides one approach to regularizing such estimates. However, standard smoothing methods ignore the fact that correlations in neural activity may fall off at different rates in different brain areas, or exhibit discontinuities across anatomical or functional boundaries. Moreover, such methods do not exploit the fact that widely separated brain regions may exhibit strong correlations due to bilateral symmetry or the network organization of brain regions. To capture this non-stationary spatial correlation structure, we introduce the brain kernel, a continuous covariance function for whole-brain activity patterns. We define the brain kernel in terms of a continuous nonlinear mapping from 3D brain coordinates to a latent embedding space, parametrized with a Gaussian process (GP). The brain kernel specifies the prior covariance between voxels as a function of the distance between their locations in embedding space. The GP mapping warps the brain nonlinearly so that highly correlated voxels are close together in latent space, and uncorrelated voxels are far apart. We estimate the brain kernel using resting-state fMRI data, and we develop an exact, scalable inference method based on block coordinate descent to overcome the challenges of high dimensionality (10-100K voxels). Finally, we illustrate the brain kernels usefulness with applications to brain decoding and factor analysis with multiple task-based fMRI datasets.

neuroscience↗

bioRxiv2021

Sharing happy stories increases interpersonal closeness mediated by enhancing interpersonal brain synchronization

Our lives revolve around sharing stories with others. Expressing emotion (i.e., happy and sad) is an essential characteristic of sharing stories and could enhance the similarity of story comprehension across speaker-listener pairs. The Emotions as Social Information Model (EASI) suggests that emotional communication may influence interpersonal closeness, but the effect of sharing emotional (happy/sad) stories on interpersonal closeness remains poorly understood. Here, one speaker watched emotional videos and communicated the content of the videos to thirty-two listeners (happy/sad/neutral group). Both speaker and listeners neural activities were recorded using EEG. After listening, we assessed the interpersonal closeness between the speaker and listeners. Compared to the sad group, sharing happy stories showed a better recall quality and a higher rating of interpersonal closeness. Meanwhile, the happy group showed higher interpersonal brain synchronization (IBS) in the prefrontal cortex (PFC) and temporal cortex (specifically TPJ) than the sad group. Moreover, such IBS mediated the relationship between the quality of sharing stories and interpersonal closeness, and happy emotion moderated this mediation model. The magnitude of IBS differentiated high interpersonal closeness from low interpersonal closeness. Exploratory analysis using support vector regression showed that the IBS could also predict the ratings of interpersonal closeness in left-out subjects. These results suggest that IBS could serve as an indicator of whether sharing emotional stories facilitate interpersonal closeness. These findings improve our understanding of sharing emotional information among individuals that guide behaviors during interpersonal interactions.

neuroscience↗

bioRxiv2021

High-order areas and auditory cortex both represent the high-level event structure of music

Recent fMRI studies of event segmentation have found that default mode regions represent high-level event structure during movie watching. In these regions, neural patterns are relatively stable during events and shift at event boundaries. Music, like narratives, contains hierarchical event structure (e.g., sections are composed of phrases). Here, we tested the hypothesis that brain activity patterns in default mode regions reflect the high-level event structure of music. We used fMRI to record brain activity from 25 participants (male and female) as they listened to a continuous playlist of 16 musical excerpts, and additionally collected annotations for these excerpts by asking a separate group of participants to mark when meaningful changes occurred in each one. We then identified temporal boundaries between stable patterns of brain activity using a hidden Markov model and compared the location of the model boundaries to the location of the human annotations. We identified multiple brain regions with significant matches to the observer-identified boundaries, including auditory cortex, medial prefrontal cortex, parietal cortex, and angular gyrus. From these results, we conclude that both higher-order and sensory areas contain information relating to the high-level event structure of music. Moreover, the higher-order areas in this study overlap with areas found in previous studies of event perception in movies and audio narratives, including regions in the default mode network. Significance StatementListening to music requires the brain to track dynamics at multiple hierarchical timescales. In our study, we had fMRI participants listen to real-world music (classical and jazz pieces) and then used an unsupervised learning algorithm (a hidden Markov model) to model the high-level event structure of music within participants brain data. This approach revealed that default mode brain regions involved in representing the high-level event structure of narratives are also involved in representing the high-level event structure of music. These findings provide converging support for the hypothesis that these regions play a domain-general role in processing events occurring over long timescales.

neuroscience↗

bioRxiv2020

Narratives: fMRI data for evaluating models of naturalistic language comprehension

The "Narratives" collection aggregates a variety of functional MRI datasets collected while human subjects listened to naturalistic spoken stories. The current release includes 345 subjects, 891 functional scans, and 27 diverse stories of varying duration totaling ~4.6 hours of unique stimuli (~43,000 words). This data collection is well-suited for naturalistic neuroimaging analysis, and is intended to serve as a benchmark for models of language and narrative comprehension. We provide standardized MRI data accompanied by rich metadata, preprocessed versions of the data ready for immediate use, and the spoken story stimuli with time-stamped phoneme- and word-level transcripts. All code and data are publicly available with full provenance in keeping with current best practices in transparent and reproducible neuroimaging.

neuroscience↗

bioRxiv2020

Thinking ahead: prediction in context as a keystone of language in humans and machines

Departing from traditional linguistic models, advances in deep learning have resulted in a new type of predictive (autoregressive) deep language models (DLMs). Using a self-supervised next-word prediction task, these models are trained to generate appropriate linguistic responses in a given context. We provide empirical evidence that the human brain and autoregressive DLMs share three fundamental computational principles as they process natural language: 1) both are engaged in continuous next-word prediction before word-onset; 2) both match their pre-onset predictions to the incoming word to calculate post-onset surprise (i.e., prediction error signals); 3) both represent words as a function of the previous context. In support of these three principles, our findings indicate that: a) the neural activity before word-onset contains context-dependent predictive information about forthcoming words, even hundreds of milliseconds before the words are perceived; b) the neural activity after word-onset reflects the surprise level and prediction error; and c) autoregressive DLM contextual embeddings capture the neural representation of context-specific word meaning better than arbitrary or static semantic embeddings. Together, our findings suggest that autoregressive DLMs provide a novel and biologically feasible computational framework for studying the neural basis of language.

neuroscience↗

bioRxiv2020

Hybrid Hyperalignment: A single high-dimensional model of shared information embedded in cortical patterns of response and functional connectivity

Shared information content is represented across brains in idiosyncratic functional topographies. Hyperalignment addresses these idiosyncrasies by using neural responses to project individuals brain data into a common model space while maintaining the geometric relationships between distinct activity patterns. The dimensions of this common model can encode any kind of functional profiles shared across individuals, such as cortical response profiles collected during a common time-locked stimulus presentation (e.g. movie viewing) or functional connectivity profiles. Performing hyperalignment with either response-based or connectivity-based input data derives transformations to project individuals neural data from anatomical space into the common model such that functional information is optimally aligned across brains. Previously, only response or connectivity profiles were used in the derivation of these transformations. In this study, we used three separate data sets collected while participants watched feature films to derive transformations representing both response-based and connectivity-based information with a single algorithm. Our new method, hybrid hyperalignment, aligns response-based information as well as or better than response hyperalignment while simultaneously aligning connectivity-based information better than connectivity hyperalignment, all in one information space. These results suggest that a single common information space could encode both shared cortical response and functional connectivity profiles across individuals.

neuroscience↗

bioRxiv2020

Teacher-student neural coupling during teaching and learning

Human communication is remarkably versatile, enabling teachers to share highly abstracted and novel information with their students. What neural processes enable such transfer of information across brains during naturalistic teaching and learning? Here, we show that during lectures, wherein information transmission is unidirectional and flows from the teacher to the student, the students brain mirrors the teachers brain and that this neural coupling is correlated with learning outcomes. A teacher was scanned in fMRI giving an oral lecture with slides on a scientific topic followed by a review lecture. Students were then scanned watching either the intact lecture and review (N = 20) or a temporally scrambled version of the lecture (N = 20). Using intersubject correlation (ISC), we observed widespread teacher-student neural coupling spanning sensory cortex and language regions along the superior temporal sulcus as well as higher-level regions including posterior medial cortex (PMC), superior parietal lobule (SPL), and dorsolateral and dorsomedial prefrontal cortex. Teacher-student alignment in higher-level areas was not observed when learning was disrupted by temporally scrambling the lecture. Moreover, teacher-student coupling in PMC was significantly correlated with learning outcomes: the more closely the students brain mirrored the teachers brain, the more the student improved between behavioral pre-learning and post-learning assessments. Together, these results suggest that the alignment of neural responses between teacher and students may underlie effective communication of complex information across brains in classroom settings. Significance statementHow is technical, non-narrative information communicated from one brain to another during teaching and learning? In this fMRI study, we show that the DMN activity of teachers and students are coupled during naturalistic teaching. This teacher-student neural coupling emerges only during intact learning and is correlated with learning outcomes. Together, these findings suggest that teacher-student neural alignment underlies effective communication during teaching.

neuroscience↗

bioRxiv2019

Robust-fit to nature: an evolutionary perspective on biological (and artificial) neural networks

Evolution is a blind fitting process by which organisms, over generations, adapt to the niches of an ever-changing environment. Does the mammalian brain use similar brute-force fitting processes to learn how to perceive and act upon the world? Recent advances in training deep neural networks has exposed the power of optimizing millions of synaptic weights to map millions of observations along ecologically relevant objective functions. This class of models has dramatically outstripped simpler, more intuitive models, operating robustly in real-life contexts spanning perception, language, and action coordination. These models do not learn an explicit, human-interpretable representation of the underlying structure of the data; rather, they use local computations to interpolate over task-relevant manifolds in a high-dimensional parameter space. Furthermore, counterintuitively, over-parameterized models, similarly to evolutionary processes, can be simple and parsimonious as they provide a versatile, robust solution for learning a diverse set of functions. In contrast to traditional scientific models, where the ultimate goal is interpretability, over-parameterized models eschew interpretability in favor of solving real-life problems or tasks. We contend that over-parameterized blind fitting presents a radical challenge to many of the underlying assumptions and practices in computational neuroscience and cognitive psychology. At the same time, this shift in perspective informs longstanding debates and establishes unexpected links with evolution, ecological psychology, and artificial life.

neuroscience↗

bioRxiv2019

Leveraging shared connectivity to aggregate heterogeneous datasets into a common response space

Connectivity hyperalignment can be used to estimate a single shared response space across disjoint datasets. We develop a connectivity-based shared response model that factorizes aggregated fMRI datasets into a single reduced-dimension shared connectivity space and subject-specific topographic transformations. These transformations resolve idiosyncratic functional topographies and can be used to project response time series into shared space. We evaluate this algorithm on a large collection of heterogeneous, naturalistic fMRI datasets acquired while subjects listened to spoken stories. Projecting subject data into shared space dramatically improves between-subject story time-segment classification and increases the dimensionality of shared information across subjects. This improvement generalizes to subjects and stories excluded when estimating the shared space. We demonstrate that estimating a simple semantic encoding model in shared space improves between-subject forward encoding and inverted encoding model performance. The shared space estimated across all datasets is distinct from the shared space derived from any particular constituent dataset; the algorithm leverages shared connectivity to yield a consensus shared space conjoining diverse story stimuli.\n\nHighlightsO_LIConnectivity SRM estimates a single shared space across subjects and stimuli\nC_LIO_LITopographic transformations resolve idiosyncrasies across individuals\nC_LIO_LIShared connectivity space enhances spatiotemporal intersubject correlations\nC_LIO_LISemantic model-based encoding and decoding improves across subjects\nC_LIO_LITransformations project into a consensus space conjoining diverse stimuli\nC_LI

neuroscience↗

bioRxiv2019

Measuring shared responses across subjects using intersubject correlation

Our capacity to jointly represent information about the world underpins our social experience. By leveraging one individuals brain activity to model anothers, we can measure shared information across brains--even in dynamic, naturalistic scenarios where an explicit response model may be unobtainable. Introducing experimental manipulations allows us to measure, for example, shared responses between speakers and listeners, or between perception and recall. In this tutorial, we develop the logic of intersubject correlation (ISC) analysis and discuss the family of neuroscientific questions that stem from this approach. We also extend this logic to spatially distributed response patterns and functional network estimation. We provide a thorough and accessible treatment of methodological considerations specific to ISC analysis, and outline best practices.

neuroscience↗

bioRxiv2018

Reliable individual differences in fine-grained cortical functional architecture

Fine-grained functional organization of cortex is not well-conserved across individuals. As a result, individual differences in cortical functional architecture are confounded by topographic idiosyncrasies--i.e., differences in functional-anatomical correspondence. In this study, we used hyperalignment to align information encoded in topographically variable patterns to study individual differences in fine-grained cortical functional architecture in a common representational space. We characterized the structure of individual differences using three common functional indices, and assessed the reliability of this structure across independent samples of data in a natural vision paradigm. Hyperalignment markedly improved the reliability of individual differences across all three indices by resolving topographic idiosyncrasies and accommodating information encoded in spatially fine-grained response patterns. Our results demonstrate that substantial individual differences in cortical functional architecture exist at fine spatial scales, but are inaccessible with anatomical normalization alone.

neuroscience↗

bioRxiv2018

Modeling semantic encoding in a common neural representational space

Encoding models for mapping voxelwise semantic tuning are typically estimated separately for each individual, limiting their generalizability. In the current report, we develop a method for estimating semantic encoding models that generalize across individuals. Functional MRI was used to measure brain responses while participants freely viewed a naturalistic audiovisual movie. Word embeddings capturing agent-, action-, object-, and scene-related semantic content were assigned to each imaging volume based on an annotation of the film. We constructed both conventional within-subject semantic encoding models and between-subject models where the model was trained on a subset of participants and validated on a left-out participant. Between-subject models were trained using cortical surface-based anatomical normalization or surface-based whole-cortex hyperalignment. We used hyperalignment to project group data into an individuals unique anatomical space via a common representational space, thus leveraging a larger volume of data for out-of-sample prediction while preserving the individuals fine-grained functional-anatomical idiosyncrasies. Our findings demonstrate that anatomical normalization degrades the spatial specificity of between-subject encoding models relative to within-subject models. Hyperalignment, on the other hand, recovers the spatial specificity of semantic tuning lost during anatomical normalization, and yields model performance exceeding that of within-subject models.

neuroscience↗

bioRxiv2018

Neural responses to naturalistic clips of behaving animals in two different task contexts

Neuroimaging studies of object and action representation often use controlled stimuli and implicitly assume that the relevant neural representational spaces are fixed and context-invariant. Here we present functional MRI data measured while participants freely viewed brief naturalistic video clips of behaving animals in two different task contexts. Participants performed a 1-back category repetition detection task requiring them to attend to either animal taxonomy or animal behavior. The data and analysis code are freely available, and have been curated according to the Brain Imaging Data Structure (BIDS) standard. We thoroughly describe the data, provide quality control metrics, and perform a searchlight classification analysis to demonstrate the potential utility of the data. These data are intended to provide a test bed for investigating how task demands alter the neural representation of complex stimuli and their semantic qualities.

neuroscience↗

bioRxiv2018

Cross-modal and non-monotonic representations of statistical regularity are encoded in local neural response patterns

Current neurobiological models assign a central role to predictive processes calibrated to environmental statistics. Neuroimaging studies examining the encoding of stimulus uncertainty have relied almost exclusively on manipulations in which stimuli were presented in a single sensory modality, and further assumed that neural responses vary monotonically with uncertainty. This has left a gap in theoretical development with respect to two core issues: i) are there cross-modal brain systems that encode input uncertainty in way that generalizes across sensory modalities, and ii) are there brain systems that track input uncertainty in a non-monotonic fashion? We used multivariate pattern analysis to address these two issues using auditory, visual and audiovisual inputs. We found signatures of cross-modal encoding in frontoparietal, orbitofrontal, and association cortices using a searchlight cross-classification analysis where classifiers trained to discriminate levels of uncertainty in one modality were tested in another modality. Additionally, we found widespread systems encoding uncertainty non-monotonically using classifiers trained to discriminate intermediate levels of uncertainty from both the highest and lowest uncertainty levels. These findings comprise the first comprehensive report of cross-modal and non-monotonic neural sensitivity to statistical regularities in the environment, and suggest that conventional paradigms testing for monotonic responses to uncertainty in a single sensory modality may have limited generalizability.

neuroscience↗