Biology subjects

Hagoort, P.

Publications and source records attributed to Hagoort, P..

35 records · Page 2Linked to original sources

bioRxiv2020

Lexical frequency and sentence context influence the brain's response to single words

Typical adults read remarkably quickly. Such fast reading is facilitated by brain processes that are sensitive to both word frequency and contextual constraints. It is debated as to whether these attributes have additive or interactive effects on language processing in the brain. We investigated this issue by analysing existing magnetoencephalography data from 99 participants reading intact and scrambled sentences. Using a cross-validated model comparison scheme, we found that lexical frequency predicted the word-by-word elicited MEG signal in a widespread cortical network, irrespective of sentential context. In contrast, index (ordinal word position) was more strongly encoded in sentence words, in left front-temporal areas. This confirms that frequency influences word processing independently of predictability, and that contextual constraints affect word-by-word brain responses. With a conservative multiple comparisons correction, only the interaction between lexical frequency and surprisal survived, in anterior temporal and frontal cortex, and not between lexical frequency and entropy, nor between lexical frequency and index. However, interestingly, the uncorrected index*frequency interaction revealed an effect in left frontal and temporal cortex that reversed in time and space for intact compared to scrambled sentences. Finally, we provide evidence to suggest that, in sentences, lexical frequency and predictability may independently influence early (<150ms) and late stages of word processing, but also interact during late stages of word processing (>150-250ms), thus helping to converge previous contradictory eye-tracking and electrophysiological literature. Current neuro-cognitive models of reading would benefit from accounting for these differing effects of lexical frequency and predictability on different stages of word processing.

neuroscience↗

bioRxiv2020

A hierarchy of linguistic predictions during natural language comprehension

Understanding spoken language requires transforming ambiguous acoustic streams into a hierarchy of representations, from phonemes to meaning. It has been suggested that the brain uses prediction to guide the interpretation of incoming input. However, the role of prediction in language processing remains disputed, with disagreement about both the ubiquity and representational nature of predictions. Here, we address both issues by analysing brain recordings of participants listening to audiobooks, and using a deep neural network (GPT-2) to precisely quantify contextual predictions. First, we establish that brain responses to words are modulated by ubiquitous, probabilistic predictions. Next, we disentangle model-based predictions into distinct dimensions, revealing dissociable signatures of syntactic, phonemic and semantic predictions. Finally, we show that high-level (word) predictions inform low-level (phoneme) predictions, supporting hierarchical predictive processing. Together, these results underscore the ubiquity of prediction in language processing, showing that the brain spontaneously predicts upcoming language at multiple levels of abstraction.

neuroscience↗

bioRxiv2019

No title, no theme: The joined neural space between speakers and listeners during production and comprehension of multi sentence discourse

Speakers and listeners usually interact in larger discourses than single words or even single sentences. The goal of the present study was to identify the neural bases reflecting how the mental representation of the situation denoted in a multi-sentence discourse (situation model) is constructed and shared between speakers and listeners. An fMRI study using a variant of the ambiguous text paradigm was designed. Speakers produced ambiguous texts in the scanner and listeners subsequently listened to these texts in different states of ambiguity: preceded by a highly informative, intermediately informative or no title at all. Conventional BOLD activation analyses in listeners, as well as inter-subject correlation analyses between the speakers and the listeners hemodynamic time courses were performed. Critically, only the processing of disambiguated, coherent discourse with an intelligible situation model representation involved (shared) activation in bilateral lateral parietal and medial prefrontal regions. This shared spatiotemporal pattern of brain activation between the speaker and the listener suggests that the process of memory retrieval in medial prefrontal regions and the binding of retrieved information in the lateral parietal cortex constitutes a core mechanism underlying the communication of complex conceptual representations.

neuroscience↗

bioRxiv2019

Biasing the perception of spoken words with tACS

Recent neuroimaging evidence suggests that the frequency of entrained oscillations in auditory cortices influences the perceived duration of speech segments, impacting word perception (Kosem et al. 2018). We further tested the causal influence of neural entrainment frequency during speech processing, by manipulating entrainment with continuous transcranial alternating current stimulation (tACS) at distinct oscillatory frequencies (3 Hz and 5.5 Hz) above the auditory cortices. Dutch participants listened to speech and were asked to report their percept of a target Dutch word, which contained a vowel with an ambiguous duration. Target words were presented either in isolation (first experiment) or at the end of spoken sentences (second experiment). We predicted that the frequency of the tACS current would influence neural entrainment and therewith how speech is perceptually sampled, leading to a perceptual over- or underestimation of the vowel duration. Experiment 1 revealed no significant result. In contrast, results from experiment 2 showed a significant effect of tACS frequency on target word perception. Faster tACS lead to more long-vowel word percepts, in line with previous findings suggesting that neural oscillations are instrumental in the temporal processing of speech. The different results from the two experiments suggest that the impact of tACS is dependent on the sensory context. tACS may have a stronger effect on spoken word perception when the words are presented in a continuous stream of speech as compared to when they are isolated, potentially because prior (stimulus-induced) entrainment of brain oscillations might be a prerequisite for tACS to be effective.

neuroscience↗

bioRxiv2019

Stimulus-modality independent activation of the brain network for language

The meaning of a sentence can be understood, whether presented in written or spoken form. Therefore it is highly probable that brain processes supporting language comprehension are at least partly independent of sensory modality. To identify where and when in the brain language processing is independent of sensory modality, we directly compared neuromagnetic brain signals of 200 human subjects (102 males) either reading or listening to sentences. We used multiset canonical correlation analysis to align individual subject data in a way that boosts those aspects of the signal that are common to all, allowing us to capture word-by-word signal variations, consistent across subjects and at a fine temporal scale. Quantifying this consistency in activation across both reading and listening tasks revealed a mostly left hemispheric cortical network. Areas showing consistent activity patterns include not only areas previously implicated in higher-level language processing, such as left prefrontal, superior & middle temporal areas and anterior temporal lobe, but also parts of the control-network as well as subcentral and more posterior temporal-parietal areas. Activity in this supramodal sentence processing network starts in temporal areas and rapidly spreads to the other regions involved. The findings do not only indicate the involvement of a large network of brain areas in supramodal language processing, but also indicate that the linguistic information contained in the unfolding sentences modulates brain activity in a word-specific manner across subjects.

neuroscience↗

bioRxiv2019

Speaking in the brain: The interaction between words and syntax in producing sentences

This neuroimaging study investigated the neural infrastructure of sentence-level language production. We compared brain activation patterns, as measured with BOLD-fMRI, during production of sentences which differed in verb argument structures (intransitives, transitives, ditransitives) and the lexical status of the verb (known verbs or pseudo-verbs). An example for the type of sentence to be produced started a mini-block of six sentences with the same structure. For each trial, participants were first given the (pseudo-)verb followed by three geometric shapes to serve as verb arguments in the sentences. Production of sentences with known verbs yielded greater activation compared to those with pseudo-verbs in the core language network of left inferior frontal gyrus, the left posterior middle temporal gyrus, and a more posterior middle temporal region extending into the angular gyrus (LpMTG/AG), analogous to effects observed in language comprehension. Increasing the number of verb arguments led to greater activation in an overlapping left pMTG/AG area, particularly for known verbs, as well as in the bilateral precuneus. Thus, producing sentences with more complex structures using existing verbs lead to increased activation in the language network, suggesting some reliance on memory retrieval of stored lexical-syntactic information during sentence production. This study thus provides evidence from sentence-level language production in line with functional models of the language network that have so far been mainly based on single word production, comprehension and processing in aphasia.

neuroscience↗

bioRxiv2019

Learning lexical-syntactic biases: An fMRI study on how we connect words and syntactic information

Language processing often involves learning new words and how they relate to each other. These relations are realized through syntactic information connected to a word, e.g. a word can be verb or a noun, or both, like the word run. In a behavioral and an fMRI task we showed that words and their syntactic properties, i.e. lexical items which were either syntactically ambiguous or unambiguous, can be learned through the probabilities of co-occurrence in an exposure session and subsequently used in a production task. Novel words were processed within regions of the language network (left inferior frontal and posterior middle temporal gyrus) and more syntactic options led to higher activations herein, even when the words were shown in isolation, suggesting combined lexical-syntactic representation. When words were shown in untrained grammatical contexts, activation in left inferior frontal cortex increased. This might reflect competition between the newly learned representation and the presented information. The results elucidate the lexical nature of the neural representations of lexical-syntactic information within the language network and the specific role of the left inferior frontal cortex in unification of the novel words with the surrounding context.

neuroscience↗

bioRxiv2019

Laminar Specific fMRI Reveals Directed Interactions in Distributed Networks During Language Processing

Laminar resolution, functional magnetic resonance imaging (lfMRI) is a noninvasive technique with the potential to distinguish top-down and bottom-up signal contributions on the basis of laminar specific interactions between distal regions. Hitherto, lfMRI could not be demonstrated for either whole-brain distributed networks or for complex cognitive tasks. We show that lfMRI can reveal whole-brain directed networks during word reading. We identify distinct, language critical regions based on their association with the top-down signal stream and establish lfMRI for the noninvasive assessment of directed connectivity during task performance.

neuroscience↗

bioRxiv2019

Supramodal sentence processing in the human brain: fMRI evidence for the influence of syntactic complexity in more than 200 participants

This study investigated two questions. One is to which degree sentence processing beyond single words is independent of the input modality (speech vs. reading). The second question is which parts of the network recruited by both modalities is sensitive to syntactic complexity. These questions were investigated by having more than 200 participants read or listen to well-formed sentences or series of unconnected words. A largely left-hemisphere fronto-temporoparietal network was found to be supramodal in nature, i.e. independent of input modality. In addition, the left inferior frontal gyrus (LIFG) and the left posterior middle temporal gyrus (LpMTG) were most clearly associated with left-branching complexity. The left anterior middle temporal gyrus (LaMTG) showed the greatest sensitivity to sentences that differed in right-branching complexity. Moreover, activity in LIFG and LpMTG increased from sentence onset to end, in parallel with an increase of the left-branching complexity. While LIFG, bilateral anterior and posterior MTG and left inferior parietal lobe (LIPL) all contribute to the supramodal unification processes, the results suggest that these regions differ in their respective contributions to syntactic complexity related processing. The consequences of these findings for neurobiological models of language processing are discussed.

neuroscience↗

bioRxiv2019

Neuronal memory for language processing

In language processing, an interpretation is computed incrementally within memory while utterances unfold in time. Here, we investigate the nature of this processing memory in a spiking network model of sentence comprehension. We show that the history dependence of neuronal responses endows circuits of biological neurons with adequate memory to assign semantic roles and resolve binding relations between words in a stream of language input. A neurobiological read-write memory is proposed where short-lived spiking activity encodes information into coupled dynamic variables that move at slower timescales. This state-dependent network does not rely on persistent activity, excitatory feedback, or synaptic plasticity for storage. Instead, information is maintained in adaptive neuronal conductances and can be accessed directly during comprehension without cued retrieval of previous input words. This work provides a step towards a computational neurobiology of language.

neuroscience↗

bioRxiv2018

The Relation between Alpha/Beta Oscillations and the Encoding of Sentence induced Contextual Information

Within the sensory domain, alpha/beta oscillations have been frequently linked to the prediction of upcoming sensory input. Here, we investigated whether oscillations at these frequency bands serve as a neural marker in the context of linguistic input prediction as well. Specifically, we hypothesized that if alpha/beta oscillations do index language prediction, their power should modulate during sentence processing, indicating stronger engagement of underlying neuronal populations involved in the linguistic prediction process. Importantly, the modulation should monotonically relate to the degrees of predictability of incoming words based on past context. Specifically, we expected that the more predictable the last word of a sentence, the stronger the alpha/beta power modulation. To test this, we measured neural responses with magnetoencephalography of healthy individuals (of either sex) during exposure to a set of linguistically matched sentences featuring three distinct levels of sentence context constraint (high, medium and low constraint). We observed fluctuations in alpha/beta power before last word onset, and also modulations in M400 amplitude after last word onset that are known to gradually relate to semantic predictability. In line with previous findings, the M400 amplitude was monotonically related to the degree of context constraint, with a high constraining context resulting in the strongest amplitude decrease. In contrast, alpha/beta power was non-monotonically related to context constraints. The strongest power decrease was observed for intermediate constraints, followed by high and low constraints. While the monotonous M400 amplitude modulation fits within a framework of prediction, the non-monotonous oscillatory results are not easily reconciled with this idea.\n\nSIGNIFICANCE STATEMENTNeural activity in the alpha (8-10Hz) and beta (16-20) frequency ranges have been related to the prediction of upcoming sensory input. It remains still debated whether these frequency bands relate to language prediction as well. In this magnetoencephalography study, we recorded alpha/beta oscillatory activity while participants listened to sentences whose ending had varying degree of predictability based on past linguistic information. Our results show that alpha/beta power modulations were non-monotonically related to the degree of linguistic predictability: the strongest modulation of alpha/beta power was observed for intermediate levels of linguistic predictability during sentence reading. Together, the results emphasize that alpha/beta oscillations cannot directly be linked to predictability in language, but potentially relate to attention or control operations during language processing.

neuroscience↗

bioRxiv2018

Do we predict upcoming speech content in naturalistic environments?

The ability to predict upcoming actions is a characteristic hallmark of cognition and therefore not surprisingly a central topic in cognitive science. It remains unclear, however, whether the predictive behaviour commonly observed in strictly controlled lab environments generalizes to rich, everyday settings. In four virtual reality experiments, we tested whether a well-established marker of linguistic prediction (i.e. anticipatory eye movements as observed in the visual world paradigm) replicated when increasing the naturalness of the paradigm by means of i) immersing participants in naturalistic everyday scenes, ii) increasing the number of distractor objects present, iii) manipulating the location of referents in central versus peripheral vision, and iv) modifying the proportion of predictable nounreferents in the experiment. Robust anticipatory eye movements were observed, even in the presence of 10 objects (hereby testing working memory) and when only 25% of all sentences contained a visually present referent (hereby testing error-based learning). The anticipatory effect disappeared, however, when referents were placed in peripheral vision. Together, our findings suggest that working memory may play an important role in predictive processing in everyday communication, but only in contexts where upcoming referents have been explicitly attended to prior to encountering the spoken referential act. Methodologically, our study confirms that ecological validity and experimental control may go hand in hand in future studies of human predictive behaviour.

neuroscience↗

bioRxiv2018

Distracting linguistic information impairs neural entrainment to attended speech

Listening to speech is difficult in noisy environments, and is even harder when the interfering noise consists of intelligible speech as compared to unintelligible sounds. This suggests that the competing linguistic information interferes with the neural processing of target speech. Interference could either arise from a degradation of the neural representation of the target speech, or from increased representation of distracting speech that enters in competition with the target speech. We tested these alternative hypotheses using magnetoencephalography (MEG) while participants listened to a target clear speech in the presence of distracting noise-vocoded speech. Crucially, the distractors were initially unintelligible but became more intelligible after a short training session. Results showed that the comprehension of the target speech was poorer after training than before training. The neural tracking of target speech in the delta range (1-4 Hz) reduced in strength in the presence of a more intelligible distractor. In contrast, the neural tracking of distracting signals was not significantly modulated by intelligibility. These results suggest that the presence of distracting speech signals degrades the linguistic representation of target speech carried by delta oscillations.

neuroscience↗

bioRxiv2017

Changes in alpha activity reveal that social opinion modulates attention allocation during face processing

AbstractParticipants performance differs when conducting a task in the presence of a secondary individual, moreover the opinion the participant has of this individual also plays a role. Using EEG, we investigated how previous interactions with, and evaluations of, an avatar in virtual reality subsequently influenced attentional allocation to the face of that avatar. We focused on changes in the alpha activity as an index of attentional allocation. We found that the onset of an avatars face whom the participant had developed a rapport with induced greater alpha suppression. This suggests greater attentional resources allocated to the interacted-with avatars. The evaluative ratings of the avatar induced a U-shaped change in alpha suppression, such that participants paid most attention when the avatar was rated as average. These results suggest that attentional allocation is thus an important element of how behaviour is altered in the presence of a secondary individual.

neuroscience↗

bioRxiv2017

Binding language: Structuring sentences through precisely timed oscillatory mechanisms

Syntactic binding refers to combining words into larger structures. Using EEG, we investigated the neural processes involved in syntactic binding. Participants were auditorily presented two-word sentences (i.e. a pronoun and pseudoverb such as she dotches, for which syntactic binding can take place) and wordlists (i.e. two pseudoverbs such as pob dotches, for which no binding can occur). Comparing these two conditions, we targeted syntactic binding while minimizing contributions of semantic binding and of other cognitive processes such as working memory. We found a converging pattern of results using two distinct analysis approaches: one approach using frequency bands as defined in previous literature, and one data-driven approach in which we looked at the entire range of frequencies between 3-30 Hz without the constraints of pre-defined frequency bands. In the syntactic binding (relative to the wordlist) condition, a power increase was observed in the alpha and beta frequency range shortly preceding the presentation of the target word that requires binding, which was maximal over frontal-central electrodes. Our interpretation is that these signatures reflect that language comprehenders expect the need for binding to occur. Following the presentation of the target word in a syntactic binding context (relative to the wordlist condition), an increase in alpha power maximal over a left lateralized cluster of frontal-temporal electrodes was observed. We suggest that this alpha increase relates to syntactic binding taking place. Taken together, our findings suggest that increases in alpha and beta power are reflections of distinct the neural processes underlying syntactic binding.

neuroscience↗

bioRxiv2017

Neural entrainment determines the words we hear

Low-frequency neural entrainment to rhythmic input has been hypothesized as a canonical mechanism that shapes sensory perception in time. Neural entrainment is deemed particularly relevant for speech analysis, as it would contribute to the extraction of discrete linguistic elements from continuous acoustic signals. Yet, its causal influence in speech perception has been difficult to establish. Here, we provide evidence that oscillations build temporal predictions about the duration of speech tokens that directly influence perception. Using magnetoencephalography (MEG), we studied neural dynamics during listening to sentences that changed in speech rate. We observed neural entrainment to preceding speech rhythms persisting for several cycles after the change in rate. The sustained entrainment was associated with changes in the perceived duration of the last words vowel, resulting in the perception of words with radically different meanings. These findings support oscillatory models of speech processing, suggesting that neural oscillations actively shape speech perception.

neuroscience↗

bioRxiv2017

Frequency-specific directed interactions in the human brain network for language

The brains remarkable capacity for language requires bidirectional interactions between functionally specialized brain regions. We used magnetoencephalography to investigate interregional interactions in the brain network for language, while 102 participants were reading sentences. Using Granger causality analysis, we identified inferior frontal cortex and anterior temporal regions to receive widespread input, and middle temporal regions to send widespread output. This fits well with the notion that these regions play a central role in language processing. Characterization of the functional topology of this network, using data-driven matrix factorization, which allowed for partitioning into a set of subnetworks, revealed directed connections at distinct frequencies of interaction. Connections originating from temporal regions peaked at alpha frequency, whereas connections originating from frontal and parietal regions peaked at beta frequency. These findings indicate that processing different types of linguistic information may depend on the contributions of distinct brain rhythms.\n\nOne Sentence SummaryCommunication between language relevant areas in the brain is supported by rhythmic synchronization, where different rhythms reflect the direction of information flow.

neuroscience↗