Biology subjects

Fedorenko, E.

Publications and source records attributed to Fedorenko, E..

At least 37 records · Page 2Linked to original sources

bioRxiv2024

Linguistic inputs must be syntactically parsable to fully engage the language network

Human language comprehension is remarkably robust to ill-formed inputs (e.g., word transpositions). This robustness has led some to argue that syntactic parsing is largely an illusion, and that incremental comprehension is more heuristic, shallow, and semantics-based than is often assumed. However, the available data are also consistent with the possibility that humans always perform rule-like symbolic parsing and simply deploy error correction mechanisms to reconstruct ill-formed inputs when needed. We put these hypotheses to a new stringent test by examining brain responses to a) stimuli that should pose a challenge for syntactic reconstruction but allow for complex meanings to be built within local contexts through associative/shallow processing (sentences presented in a backward word order), and b) grammatically well-formed but semantically implausible sentences that should impede semantics-based heuristic processing. Using a novel behavioral syntactic reconstruction paradigm, we demonstrate that backward- presented sentences indeed impede the recovery of grammatical structure during incremental comprehension. Critically, these backward-presented stimuli elicit a relatively low response in the language areas, as measured with fMRI. In contrast, semantically implausible but grammatically well-formed sentences elicit a response in the language areas similar in magnitude to naturalistic (plausible) sentences. In other words, the ability to build syntactic structures during incremental language processing is both necessary and sufficient to fully engage the language network. Taken together, these results provide strongest to date support for a generalized reliance of human language comprehension on syntactic parsing. Significance statementWhether language comprehension relies predominantly on structural (syntactic) cues or meaning- related (semantic) cues remains debated. We shed new light on this question by examining the language brain areas responses to stimuli where syntactic and semantic cues are pitted against each other, using fMRI. We find that the language areas respond weakly to stimuli that allow for local semantic composition but cannot be parsed syntactically--as confirmed in a novel behavioral paradigm--and they respond strongly to grammatical but semantically implausible sentences, like the famous Colorless green ideas sleep furiously sentence. These findings challenge accounts of language processing that suggest that syntactic parsing can be foregone in favor of shallow semantic processing.

neuroscience↗

bioRxiv2024

Precision fMRI reveals that the language network exhibits adult-like left-hemispheric lateralization by 4 years of age

In adults, left hemisphere (LH) damage often leads to aphasia, but many cases of early damage leave linguistic processing intact, with a functional language system developing in the right hemisphere. To explain this early apparent equipotentiality of the two hemispheres for language, some have proposed that the language system is more bilateral during early development and becomes increasingly left-lateralized with age. We examined language lateralization using fMRI in two large developmental cohorts (total n=273 children aged 4-16 years; n=107 adults). Strong, adult-like LH lateralization (in response magnitude and activation volume) was evident by age 4, although other features of the LH language network showed protracted development, including the magnitude of language response and the strength of functional connectivity. Thus, although the RH can take over language function in some cases of early brain damage, this plasticity occurs in spite of adult-level LH bias present by age 4 years.

neuroscience↗

bioRxiv2023

Large language models implicitly learn to straighten neural sentence trajectories to construct a predictive representation of natural language

Predicting upcoming events is critical to our ability to effectively interact with our environment and conspecifics. In natural language processing, transformer models, which are trained on next-word prediction, appear to construct a general-purpose representation of language that can support diverse downstream tasks. However, we still lack an understanding of how a predictive objective shapes such representations. Inspired by recent work in vision neuroscience Henaff et al. (2019), here we test a hypothesis about predictive representations of autoregressive transformer models. In particular, we test whether the neural trajectory of a sequence of words in a sentence becomes progressively more straight as it passes through the layers of the network. The key insight behind this hypothesis is that straighter trajectories should facilitate prediction via linear extrapolation. We quantify straightness using a 1-dimensional curvature metric, and present four findings in support of the trajectory straightening hypothesis: i) In trained models, the curvature progressively decreases from the first to the middle layers of the network. ii) Models that perform better on the next-word prediction objective, including larger models and models trained on larger datasets, exhibit greater decreases in curvature, suggesting that this improved ability to straighten sentence neural trajectories may be the underlying driver of better language modeling performance. iii) Given the same linguistic context, the sequences that are generated by the model have lower curvature than the ground truth (the actual continuations observed in a language corpus), suggesting that the model favors straighter trajectories for making predictions. iv) A consistent relationship holds between the average curvature and the average surprisal of sentences in the middle layers of models, such that sentences with straighter neural trajectories also have lower surprisal. Importantly, untrained models dont exhibit these behaviors. In tandem, these results support the trajectory straightening hypothesis and provide a possible mechanism for how the geometry of the internal representations of autoregressive models supports next word prediction.

neuroscience↗

bioRxiv2023

Constructed languages are processed by the same brain mechanisms as natural languages

What constitutes a language? Natural languages share features with other domains: from math, to music, to gesture. However, the brain mechanisms that process linguistic input are highly specialized, showing little response to diverse non-linguistic tasks. Here, we examine constructed languages (conlangs) to ask whether they draw on the same neural mechanisms as natural languages, or whether they instead pattern with domains like math and programming languages. Using individual-subject fMRI analyses, we show that understanding conlangs recruits the same brain areas as natural language comprehension. This result holds for Esperanto (n=19 speakers) and four fictional conlangs (Klingon (n=10), Navi (n=9), High Valyrian (n=3), and Dothraki (n=3)). These findings suggest that conlangs and natural languages share critical features that allow them to draw on the same representations and computations, implemented in the left-lateralized network of brain areas. The features of conlangs that differentiate them from natural languages--including recent creation by a single individual, often for an esoteric purpose, the small number of speakers, and the fact that these languages are typically learned in adulthood-- appear to not be consequential for the reliance on the same cognitive and neural mechanisms. We argue that the critical shared feature of conlangs and natural languages is that they are symbolic systems capable of expressing an open-ended range of meanings about our outer and inner worlds. Significance StatementWhat constitutes a language has been of interest to diverse disciplines - from philosophy and linguistics to psychology, anthropology, and sociology. An empirical approach is to test whether the system in question recruits the brain system that processes natural languages. In spite of their similarity to natural languages, math and programming languages recruit a distinct brain system. Using fMRI, we test brain responses to stimuli not previously investigated--constructed languages (conlangs)--and find that they are processed by the same brain network as natural languages. Thus, an ability for a symbolic system to express diverse meanings about the world-- but not the recency, manner, and purpose of its creation, or a large user base--is a defining characteristic of a language.

neuroscience↗

bioRxiv2023

Lexical semantic content, not syntactic structure, is the main contributor to ANN-brain similarity of fMRI responses in the language network

Representations from artificial neural network (ANN) language models have been shown to predict human brain activity in the language network. To understand what aspects of linguistic stimuli contribute to ANN-to-brain similarity, we used an fMRI dataset of responses to n=627 naturalistic English sentences (Pereira et al., 2018) and systematically manipulated the stimuli for which ANN representations were extracted. In particular, we i) perturbed sentences word order, ii) removed different subsets of words, or iii) replaced sentences with other sentences of varying semantic similarity. We found that the lexical semantic content of the sentence (largely carried by content words) rather than the sentences syntactic form (conveyed via word order or function words) is primarily responsible for the ANN-to-brain similarity. In follow-up analyses, we found that perturbation manipulations that adversely affect brain predictivity also lead to more divergent representations in the ANNs embedding space and decrease the ANNs ability to predict upcoming tokens in those stimuli. Further, results are robust to whether the mapping model is trained on intact or perturbed stimuli, and whether the ANN sentence representations are conditioned on the same linguistic context that humans saw. The critical result--that lexical- semantic content is the main contributor to the similarity between ANN representations and neural ones--aligns with the idea that the goal of the human language system is to extract meaning from linguistic strings. Finally, this work highlights the strength of systematic experimental manipulations for evaluating how close we are to accurate and generalizable models of the human language network.

neuroscience↗

bioRxiv2023

Driving and suppressing the human language network using large language models

Transformer models such as GPT generate human-like language and are highly predictive of human brain responses to language. Here, using fMRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict the magnitude of brain response associated with each sentence. Then, we use the model to identify new sentences that are predicted to drive or suppress responses in the human language network. We show that these model-selected novel sentences indeed strongly drive and suppress activity of human language areas in new individuals. A systematic analysis of the model-selected sentences reveals that surprisal and well-formedness of linguistic input are key determinants of response strength in the language network. These results establish the ability of neural network models to not only mimic human language but also noninvasively control neural activity in higher-level cortical areas, like the language network.

neuroscience↗

bioRxiv2023

Functional identification of language-responsive channels in individual participants in MEG investigations

Making meaningful inferences about the functional architecture of the language system requires the ability to refer to the same neural units across individuals and studies. Traditional brain imaging approaches align and average brains together in a common space. However, lateral frontal and temporal cortices, where the language system resides, is characterized by high structural and functional inter-individual variability, which reduces the sensitivity and functional resolution of group-averaging analyses. This issue is compounded by the fact that language areas lay in close proximity to regions of other large-scale networks with different functional profiles. A solution inspired by visual neuroscience is to identify language areas functionally in each individual brain using a localizer task (e.g., a language comprehension task). This approach has proven productive in fMRI, yielding a number of robust and replicable findings about the language system. Here, we extend this approach to MEG. Across two experiments (one in Dutch speakers, n=19; one in English speakers, n=23), we examined neural responses to the processing of sentences and a control condition (nonword sequences). In both the time and frequency domains, we demonstrated that the topography of neural responses to language is spatially stable within individuals but varies across individuals. Consequently, analyses that take this inter-individual variability into account are characterized by greater sensitivity, compared to the group-level analyses. In summary, similar to fMRI, functional identification within individuals yields benefits in MEG, thus opening the door to future investigations of language processing including questions where whole-brain coverage and temporal resolution are both critical.

neuroscience↗

bioRxiv2023

Functional characterization of the language network of polyglots and hyperpolyglots with precision fMRI

How do polyglots--individuals who speak five or more languages--process their languages, and what can this population tell us about the language system? Using fMRI, we identified the language network in each of 34 polyglots (including 16 hyperpolyglots with knowledge of 10+ languages) and examined its response to the native language, non-native languages of varying proficiency, and unfamiliar languages. All language conditions engaged all areas of the language network relative to a control condition. Languages that participants rated as higher-proficiency elicited stronger responses, except for the native language, which elicited a similar or lower response than a non-native language of similar proficiency. Furthermore, unfamiliar languages that were typologically related to the participants high-to-moderate-proficiency languages elicited a stronger response than unfamiliar unrelated languages. The results suggest that the language networks response magnitude scales with the degree of engagement of linguistic computations (e.g., related to lexical access and syntactic-structure building). We also replicated a prior finding of weaker responses to native language in polyglots than non-polyglot bilinguals. These results contribute to our understanding of how multiple languages co-exist within a single brain and provide new evidence that the language network responds more strongly to stimuli that more fully engage linguistic computations.

neuroscience↗

bioRxiv2023

Preserved functional organization of human auditory cortex in individuals missing one temporal lobe from infancy

Human cortical responses to natural sounds, measured with fMRI, can be approximated as the weighted sum of a small number of canonical response patterns (components), each having interpretable functional and anatomical properties. Here, we asked whether this organization is preserved in cases where only one temporal lobe is available due to early brain damage by investigating a unique family: one sibling born without a left temporal lobe, another without a right temporal lobe, and a third anatomically neurotypical. We analyzed fMRI responses to diverse natural sounds within the intact hemispheres of these individuals and compared them to 12 neurotypical participants. All siblings manifested the neurotypical auditory responses in their intact hemispheres. These results suggest that the development of the auditory cortex in each hemisphere does not depend on the existence of the other hemisphere, highlighting the redundancy and equipotentiality of the bilateral auditory system.

neuroscience↗

bioRxiv2022

Intracranial recordings reveal three distinct neural response patterns in the language network

Despite long knowing what brain areas support language comprehension, our knowledge of the neural computations that these frontal and temporal regions implement remains limited. One important unresolved question concerns functional differences among the neural populations that comprise the language network. Leveraging the high spatiotemporal resolution of intracranial recordings, we examined responses to sentences and linguistically degraded conditions and discovered three response profiles that differ in their temporal dynamics. These profiles appear to reflect different temporal receptive windows (TRWs), with average TRWs of about 1, 4, and 6 words, as estimated with a simple one-parameter model. Neural populations exhibiting these profiles are interleaved across the language network, which suggests that all language regions have direct access to distinct, multi-scale representations of linguistic input--a property that may be critical for the efficiency and robustness of language processing.

neuroscience↗

bioRxiv2022

Artificial neural network language models align neurally and behaviorally with humans even after a developmentally realistic amount of training

Artificial neural networks have emerged as computationally plausible models of human language processing. A major criticism of these models is that the amount of training data they receive far exceeds that of humans during language learning. Here, we use two complementary approaches to ask how the models ability to capture human fMRI responses to sentences is affected by the amount of training data. First, we evaluate GPT-2 models trained on 1 million, 10 million, 100 million, or 1 billion words against an fMRI benchmark. We consider the 100-million-word model to be developmentally plausible in terms of the amount of training data given that this amount is similar to what children are estimated to be exposed to during the first 10 years of life. Second, we test the performance of a GPT-2 model trained on a 9-billion-token dataset to reach state-of-the-art next-word prediction performance on the human benchmark at different stages during training. Across both approaches, we find that (i) the models trained on a developmentally plausible amount of data already achieve near-maximal performance in capturing fMRI responses to sentences. Further, (ii) lower perplexity--a measure of next-word prediction performance--is associated with stronger alignment with human data, suggesting that models that have received enough training to achieve sufficiently high next-word prediction performance also acquire representations of sentences that are predictive of human fMRI responses. In tandem, these findings establish that although some training is necessary for the models predictive ability, a developmentally realistic amount of training ([~]100 million words) may suffice.

neuroscience↗

bioRxiv2022

No evidence of theory of mind reasoning in the human language network

Language comprehension and the ability to infer others thoughts (theory of mind, ToM) are interrelated during development and language use. However, neural evidence that bears on the relationship between language and ToM mechanisms is mixed. Although robust dissociations have been reported in brain disorders, brain activations for contrasts that target language and ToM bear similarities, and some have reported overlap (Deen et al., 2015). We take another look at the language-ToM relationship by evaluating the response of the language network (Fedorenko et al., 2010), as measured with fMRI, to verbal and non-verbal ToM across 151 participants. Individual-subject analyses reveal that all core language regions respond more strongly when participants read vignettes about false beliefs compared to the control vignettes. However, we show that these differences are largely due to linguistic confounds, and no such effects appear in a non-verbal ToM task. These results argue against cognitive and neural overlap between language processing and ToM. In exploratory analyses, we find responses to social processing in the "periphery" of the language network--right hemisphere homotopes of core language areas and areas in bilateral angular gyri--but these responses are not selectively ToM-related and may reflect general visual semantic processing.

neuroscience↗

bioRxiv2022

The language network reliably 'tracks' naturalistic meaningful non-verbal stimuli

The language network, comprised of brain regions in the left frontal and temporal cortex, responds robustly and reliably during language comprehension but shows little or no response during many non-linguistic cognitive tasks (e.g., Fedorenko & Blank, 2020). However, one domain whose relationship with language remains debated is semantics--our conceptual knowledge of the world. Given that the language network responds strongly to meaningful linguistic stimuli, could some of this response be driven by the presence of rich conceptual representations encoded in linguistic inputs? In this study, we used a naturalistic cognition paradigm to test whether the cognitive and neural resources that are responsible for language processing are also recruited for processing semantically rich non-verbal stimuli. To do so, we measured BOLD responses to a set of [~]5-minute-long video and audio clips that consisted of meaningful event sequences but did not contain any linguistic content. We then used the inter-subject correlation (ISC) approach (Hasson et al., 2004) to examine the extent to which the language network tracks these stimuli, i.e. exhibits stimulus-related variation. Across all the regions of the language network, non-verbal meaningful stimuli elicited reliable ISCs. These ISCs were higher than the ISCs elicited by semantically impoverished non-verbal stimuli (e.g., a music clip), but substantially lower than the ISCs elicited by linguistic stimuli. Our results complement earlier findings from controlled experiments (e.g., Ivanova et al., 2021) in providing further evidence that the language network shows some sensitivity to semantic content in non-verbal stimuli.

neuroscience↗

bioRxiv2022

Causal contributions of the domain-general (Multiple Demand) and the language-selective brain networks to perceptual and semantic challenges in speech comprehension

1.Listening to spoken language engages domain-general Multiple Demand (MD, fronto-parietal) regions of the human brain, in addition to domain-selective (fronto-temporal) language regions, particularly when comprehension is challenging. However, there is limited evidence that the MD network makes a functional contribution to core aspects of comprehension. In a behavioural study of volunteers (n=19) with chronic brain lesions, but without aphasia, we assessed the causal role of these networks in perceiving, comprehending and adapting to challenging spoken sentences. A first task measured word report for acoustically degraded (noise-vocoded) sentences before and after training. Participants with greater damage to MD but not language regions required more vocoder channels to achieve 50% word report indicating impaired perception. Perception improved following training, reflecting adaptation to acoustic degradation, but perceptual learning was unrelated to lesion location or extent. A second task used sentence coherence judgements to measure the speed and accuracy of comprehension of spoken sentences using lower-frequency meanings of semantically ambiguous words. Comprehension accuracy was high and unaffected by lesion location or extent. The availability of the lower-frequency meaning, as measured in a subsequent word association task, increased following comprehension (word-meaning priming). Word-meaning priming was reduced for participants with greater damage to language but not MD regions. We conclude that language and MD networks make dissociable contributions to challenging speech comprehension: using recent experience to update word meaning preferences depends on language specialised regions, whereas the domain-general MD network plays a causal role in reporting words from degraded speech.

neuroscience↗

bioRxiv2022

Non-literal language processing is jointly supported by the language and Theory of Mind networks: Evidence from a novel meta-analytic fMRI approach

Going beyond the literal meaning of utterances is key to communicative success. However, the mechanisms that support non-literal inferences remain debated. Using a novel meta-analytic approach, we evaluate the contribution of linguistic, social-cognitive, and executive mechanisms to non-literal interpretation. We identified 74 fMRI experiments (n=1,430 participants) from 2001-2021 that contrasted non-literal language comprehension with a literal control condition, spanning ten phenomena (e.g., metaphor, irony, indirect speech). Applying the activation likelihood estimation approach to the 825 activation peaks yielded six left-lateralized clusters. We then evaluated the locations of both the individual-study peaks and the clusters against probabilistic functional atlases (cf. macroanatomy, as is typically done) for three candidate brain networks--the language-selective network (Fedorenko et al., 2011), which supports language processing, the Theory of Mind (ToM) network (Saxe & Kanwisher, 2003), which supports social inferences, and the domain-general Multiple-Demand (MD) network (Duncan, 2010), which supports executive control. These atlases were created by overlaying individual activation maps of participants who performed robust and extensively validated localizer tasks that target each network in question (n=806 for language; n=198 for ToM; n=691 for MD). We found that both the individual-study peaks and the ALE clusters fell primarily within the language network and the ToM network. These results suggest that non-literal processing is supported by both i) mechanisms that process literal linguistic meaning, and ii) mechanisms that support general social inference. They thus undermine a strong divide between literal and non-literal aspects of language and challenge the claim that non-literal processing requires additional executive resources.

neuroscience↗

bioRxiv2022

LanA (Language Atlas): A probabilistic atlas for the language network based on fMRI data from >800 individuals

Two analytic traditions characterize fMRI language research. One relies on averaging activations voxel-wise across individuals. This approach has limitations: because of inter-individual variability in the locations of language areas, a location in a common brain space cannot be meaningfully linked to function. An alternative approach relies on identifying language areas in each individual using a functional localizer. Because of its greater sensitivity, functional resolution, and interpretability, functional localization is gaining popularity, but it is not always feasible, and cannot be applied retroactively to past studies. We provide a solution for bridging these currently disjoint approaches in the form of a probabilistic functional atlas created from fMRI data for an extensively validated language localizer in 806 individuals. This atlas enables estimating the probability that any given location in a common brain space belongs to the language network, and thus can help interpret group-level peaks and meta-analyses of such peaks, and lesion locations in patient investigations. More meaningful comparisons of findings across studies should increase robustness and replicability in language research.

neuroscience↗

bioRxiv2021

'Constituent length' effects in fMRI do not provide evidence for abstract syntactic processing

Human language has a remarkable capacity to encode complex ideas. This capacity arises because language is compositional: the form and arrangement of words in sentences (structure) determine the conceptual relations that hold between the words referents (meaning). A foundational question in human cognition is whether the brain regions that support language are similarly factored into structure-selective and meaning-selective areas. In an influential study, Pallier et al. (2011, PNAS) used fMRI to investigate the brain response to sequences of real words and pseudowords and reported a sharp dissociation between structure-selective and meaning-selective brain regions. In the present study, we argue that no such dissociation emerges when individual differences in brain anatomy are considered. We report three experiments (including a close conceptual replication of Pallier et al.s original study) that use precision fMRI methods to capture separation or overlap of function in the brains of individual participants. Our results replicate Pallier et al.s finding that the brains response is modulated by the sequential structure of language but paint a different picture with respect to the structure-meaning relationship. Instead of distinct structure-selective and meaning-selective brain areas, we find distributed sensitivity to both linguistic structure and meaning throughout a broad frontotemporal brain network. Our results join a growing body of evidence for an integrated network for language in the human brain within which internal specialization is primarily a matter of degree rather than kind, in contrast with influential proposals that advocate distinct specialization of different brain areas for different types of linguistic functions. Significance StatementUsing fMRI, we show that a broad network of frontal and temporal areas in the left hemisphere of the human brain is sensitive to both the structure of language and the meaning that it encodes. This finding challenges many current theories of the neurobiology of language, which propose a sharp separation between areas that encode structure and areas that encode meaning. Instead, results support a broad distribution of word- and sentence-level processing across an integrated brain network for language. This PDF file includes: Main Text Figures 1 to 3 Tables 1 to 1

neuroscience↗

bioRxiv2021

Robust effects of working memory demand during naturalistic language comprehension in language-selective cortex

A standard view of human language processing is that comprehenders build richly structured mental representations of natural language utterances, word by word, using computationally costly memory operations supported by domain-general working memory resources. However, three core claims of this view have been questioned, with some prior work arguing that (1) rich word-by-word structure building is not a core function of the language comprehension system, (2) apparent working memory costs are underlyingly driven by word predictability (surprisal), and/or (3) language comprehension relies primarily on domain-general rather than domain-specific working memory resources. In this work, we simultaneously evaluate all three of these claims using naturalistic comprehension in fMRI. In each participant, we functionally localize (a) a language-selective network and (b) a multiple-demand network that supports working memory across domains, and we analyze the responses in these two networks of interest during naturalistic story listening with respect to a range of theory-driven predictors of working memory demand under rigorous surprisal controls. Results show robust surprisal-independent effects of word-by-word memory demand in the language network and no effect of working memory demand in the multiple demand network. Our findings thus support the view that language comprehension (1) entails word-by-word structure building using (2) computationally intensive memory operations that are not explained by surprisal. However, these results challenge (3) the domain-generality of the resources that support these operations, instead indicating that working memory operations for language comprehension are carried out by the same neural resources that store linguistic knowledge. Significance StatementThis study uses fMRI to investigate signatures of working memory (WM) demand during naturalistic story listening, using a broad range of theoretically motivated estimates of WM demand. Results support a strong effect of WM demand in language-selective brain regions but no effect of WM demand in "multiple demand" regions that have previously been associated with WM in non-linguistic domains. We further show evidence that WM effects in language regions are distinct from effects of word predictability. Our findings support a core role for WM in incremental language processing, using WM resources that are specialized for language.

neuroscience↗