Biology subjects

Bosker, H. R.

Publications and source records attributed to Bosker, H. R..

4 recordsLinked to original sources

bioRxiv2020

Beat gestures influence which speech sounds you hear

Beat gestures - spontaneously produced biphasic movements of the hand - are among the most frequently encountered co-speech gestures in human communication. They are closely temporally aligned to the prosodic characteristics of the speech signal, typically occurring on lexically stressed syllables. Despite their prevalence across speakers of the worlds languages, how beat gestures impact spoken word recognition is unclear. Can these simple flicks of the hand influence speech perception? Across six experiments, we demonstrate that beat gestures influence the explicit and implicit perception of lexical stress (e.g., distinguishing OBject from obJECT), and in turn, can influence what vowels listeners hear. Thus, we provide converging evidence for a manual McGurk effect: even the simplest flicks of the hands influence which speech sounds we hear. SIGNIFICANCE STATEMENTBeat gestures are very common in human face-to-face communication. Yet we know little about their behavioral consequences for spoken language comprehension. We demonstrate that beat gestures influence the explicit and implicit perception of lexical stress, and, in turn, can even shape what vowels we think we hear. This demonstration of a manual McGurk effect provides some of the first empirical support for a recent multimodal, situated psycholinguistic framework of human communication, while challenging current models of spoken word recognition that do not yet incorporate multimodal prosody. Moreover, it has the potential to enrich human-computer interaction and improve multimodal speech recognition systems.

neuroscience↗

bioRxiv2020

Structure and meaning organize neural oscillations into a content-specific hierarchy

Neural oscillations track linguistic information during speech comprehension (e.g., Ding et al., 2016; Keitel et al., 2018), and are known to be modulated by acoustic landmarks and speech intelligibility (e.g., Zoefel & VanRullen, 2015). But, it is unclear what information (e.g., timing, rhythm, or content) the brain utilizes to generate linguistic structure and meaning beyond the information that is present in the physical stimulus. We used electroencephalography (EEG) to investigate whether oscillations are modulated by linguistic content over and above the speech stimulus rhythmicity and temporal distribution. We manipulated the presence of semantic and syntactic information apart from the timescale of their occurrence, and controlled for the acoustic-prosodic and lexical-semantic information in the signal. EEG was recorded while 29 adult native speakers of all genders listened to naturally-spoken Dutch sentences, jabberwocky controls with a sentence-like prosodic rhythm and morphemes, word lists with lexical content but no phrase structure, and backwards acoustically-matched controls. Mutual information (MI) analysis revealed sensitivity to linguistic content: Phase MI was highest for sentences at the phrasal (0.8-1.1 Hz) and lexical timescale (1.9-2.8 Hz), suggesting that the delta-band is modulated by lexically-driven combinatorial processing beyond prosody, and that linguistic content (i.e., structure and meaning) organizes the phase of neural oscillations beyond the timescale and rhythmicity of the stimulus. This pattern is consistent with neurophysiologically-inspired models of language comprehension (Martin, 2016, 2020; Martin & Doumas, 2017) where oscillations encode endogenously-generated linguistic content over and above exogenous or stimulus-driven timing and rhythm information. Significance StatementBiological systems like the brain encode their environment not only by reacting in a series of stimulus-driven responses, but by combining stimulus-driven information with endogenous, internally-generated, inferential knowledge and meaning. Understanding language from speech is the human benchmark for this. Much research focusses on the purely stimulus-driven response, but here, we focus on the goal of language behavior: conveying structure and meaning. To that end, we use naturalistic stimuli that contrast acoustic-prosodic and lexical-semantic information to show that, during spoken language comprehension, oscillatory modulations reflect computations related to inferring structure and meaning from the acoustic signal. Our experiment provides the first evidence to date that compositional structure and meaning organize the oscillatory response, above and beyond acoustic and lexical controls.

neuroscience↗

bioRxiv2019

Biasing the perception of spoken words with tACS

Recent neuroimaging evidence suggests that the frequency of entrained oscillations in auditory cortices influences the perceived duration of speech segments, impacting word perception (Kosem et al. 2018). We further tested the causal influence of neural entrainment frequency during speech processing, by manipulating entrainment with continuous transcranial alternating current stimulation (tACS) at distinct oscillatory frequencies (3 Hz and 5.5 Hz) above the auditory cortices. Dutch participants listened to speech and were asked to report their percept of a target Dutch word, which contained a vowel with an ambiguous duration. Target words were presented either in isolation (first experiment) or at the end of spoken sentences (second experiment). We predicted that the frequency of the tACS current would influence neural entrainment and therewith how speech is perceptually sampled, leading to a perceptual over- or underestimation of the vowel duration. Experiment 1 revealed no significant result. In contrast, results from experiment 2 showed a significant effect of tACS frequency on target word perception. Faster tACS lead to more long-vowel word percepts, in line with previous findings suggesting that neural oscillations are instrumental in the temporal processing of speech. The different results from the two experiments suggest that the impact of tACS is dependent on the sensory context. tACS may have a stronger effect on spoken word perception when the words are presented in a continuous stream of speech as compared to when they are isolated, potentially because prior (stimulus-induced) entrainment of brain oscillations might be a prerequisite for tACS to be effective.

neuroscience↗

bioRxiv2017

Neural entrainment determines the words we hear

Low-frequency neural entrainment to rhythmic input has been hypothesized as a canonical mechanism that shapes sensory perception in time. Neural entrainment is deemed particularly relevant for speech analysis, as it would contribute to the extraction of discrete linguistic elements from continuous acoustic signals. Yet, its causal influence in speech perception has been difficult to establish. Here, we provide evidence that oscillations build temporal predictions about the duration of speech tokens that directly influence perception. Using magnetoencephalography (MEG), we studied neural dynamics during listening to sentences that changed in speech rate. We observed neural entrainment to preceding speech rhythms persisting for several cycles after the change in rate. The sustained entrainment was associated with changes in the perceived duration of the last words vowel, resulting in the perception of words with radically different meanings. These findings support oscillatory models of speech processing, suggesting that neural oscillations actively shape speech perception.

neuroscience↗