Biology subjects

Roesch, S.

Publications and source records attributed to Roesch, S..

7 recordsLinked to original sources

bioRxiv2024

Eye Movements in Silent Visual Speech track Unheard Acoustic Signals and Relate to Hearing Experience

Behavioral and neuroscientific studies have shown that watching a speakers lip movements aids speech comprehension. Intriguingly, even when videos of speakers are presented silently, various cortical regions track auditory features, such as the envelope. Recently, we demonstrated that eye movements track low-level acoustic information when attentively listening to speech. In this study we investigated whether ocular speech tracking occurs during visual speech and how it influences cortical silent speech tracking. Furthermore, we compared the data of hearing individuals with congenitally deaf individuals, and those with acquired deafness or hearing loss (DHH; Deaf or hard of hearing) to assess how auditory deprivation (early vs. late onset) affects neural and ocular speech tracking during silent lip-reading. Using magnetoencephalography (MEG), we examined ocular and neural speech tracking of 75 participants observing silent videos of a speaker played forward and backward. Our main finding is a clear ocular unheard speech tracking effect with a dominance <1 Hz, which was not present for the lip movements. Similarly, we observed a <=1.3 Hz effect of neural unheard speech tracking in temporal regions for hearing participants. Importantly, neural tracking was not directly linked to ocular tracking in this study. Strikingly, across different listening groups, deaf participants with auditory listening experience showed higher ocular speech tracking than hearing participants, while no ocular speech tracking effect was revealed for congenitally deaf participants in a very small sample. This study extends our previous work by demonstrating the involvement of eye movements in speech processing, even in the absence of acoustic input.

neuroscience↗

bioRxiv2023

Eye movements track prioritized auditory features in selective attention to natural speech

Over the last decades, cognitive neuroscience has identified a distributed set of brain regions that are critical for attention - one of the key principles of adaptive behavior. A strong anatomical overlap with brain regions critical for oculomotor processes suggests a joint network for attention and eye movements. However, the role of this shared network in complex, naturalistic environments remains understudied. Here, we investigated eye movements in relation to (un)attended sentences of natural speech in simultaneously recorded eye tracking and magnetoencephalographic (MEG) data. Using temporal response functions (TRF), we show that eye gaze tracks acoustic features (envelope and acoustic onsets) of attended speech, a phenomenon we termed ocular speech tracking. Ocular speech envelope tracking even differentiates a target from a distractor in a multi speaker context and is further related to intelligibility. Moreover, we provide evidence for its contribution to neural differences in speech processing, emphasizing the necessity to consider oculomotor activity in future research and in the interpretation of neural differences in auditory cognition. Our results extend previous findings of a joint network of attention and eye movement control as well as motor theories of speech. They provide valuable new directions for research into the neurobiological mechanisms of the phenomenon, its dependence on learning and plasticity, and its functional implications in social communication.

neuroscience↗

bioRxiv2022

Speech intelligibility changes the temporal evolution of neural speech tracking

Listening to speech with poor signal quality is challenging. Neural speech tracking of degraded speech has been used to advance the understanding of how brain processes and speech intelligibility are interrelated, however the temporal dynamics of neural speech tracking are not clear. In the present MEG study, we thereby exploited temporal response functions (TRFs) and generated signal-degraded speech to depict the temporal evolution of speech intelligibility modulation on neural speech tracking. In addition, we inter-related facets of neural speech tracking (e.g., speech envelope reconstruction, speech-brain coherence, and components of broadband coherence spectra) to endorse our findings in TRFs. Our TRF analysis yielded marked temporally differential effects of vocoding: reduction of intelligibility went along with large increases of early peak responses ([~]50-110 ms, M50TRF), but strongly reduced responses around 175-230 ms (M200TRF). For the late responses 315-380 ms (M350TRF), the maximum response occurred for degraded speech that was still comprehensible then declined with reduced intelligibility. Furthermore, we related the TRF components to our other neural "tracking" measures and found that M50TRF and M200TRF play a differential role in the shifting center frequency of the broadband coherence spectra. Overall, our study highlights the importance of time-resolved computation and parametrization of coherence spectra on neural speech tracking and provides a better understanding of degraded speech processing. HighlightsO_LIWe use MEG to show that speech intelligibility differentially impacts the temporal evolution of neural speech tracking. C_LIO_LITRF responses around 200 ms show the strongest relationship with behaviour. C_LIO_LIRelating TRF effects to parameterized coherence spectra using FOOOF suggests that M50TRF and M200TRF reflect shifts in which speech features are tracked over time. C_LI

neuroscience↗

bioRxiv2021

Age-related decreases of cortical visuo-phonological transformation of unheard spectral fine-details

The integration of visual and auditory cues is crucial for successful processing of speech, especially under adverse conditions. Recent reports have shown that when participants watch muted videos of speakers, the phonological information about the acoustic speech envelope is tracked by the visual cortex. However, the speech signal also carries much richer acoustic details, e.g. about the fundamental frequency and the resonant frequencies, whose visuo-phonological transformation could aid speech processing. Here, we investigated the neural basis of the visuo-phonological transformation processes of these more fine-grained acoustic details and assessed how they change with ageing. We recorded whole-head magnetoencephalography (MEG) data while participants watched silent intelligible and unintelligible videos of a speaker. We found that the visual cortex is able to track the unheard intelligible modulations of resonant frequencies and the pitch linked to lip movements. Importantly, only the processing of intelligible unheard formants decreases significantly with age in the visual and also in the cingulate cortex. This is not the case for the processing of the unheard speech envelope, the fundamental frequency or the purely visual information carried by lip movements. These results show that unheard spectral fine-details (along with the unheard acoustic envelope) are transformed from a mere visual to a phonological representation. Aging affects especially the ability to derive spectral dynamics at formant frequencies. Since listening in noisy environments should capitalize on the ability to track spectral fine-details, our results provide a novel focus on compensatory processes in such challenging situations.

neuroscience↗

bioRxiv2021

Neural speech tracking shifts from the syllabic to the modulation rate of speech as intelligibility decreases

The most prominent acoustic features in speech are intensity modulations, represented by the amplitude envelope of speech. Synchronization of neural activity with these modulations is vital for speech comprehension. As the acoustic modulation of speech is related to the production of syllables, investigations of neural speech tracking rarely distinguish between lower-level acoustic (envelope modulation) and higher-level linguistic (syllable rate) information. Here we manipulated speech intelligibility using noise-vocoded speech and investigated the spectral dynamics of neural speech processing, across two studies at cortical and subcortical levels of the auditory hierarchy, using magnetoencephalography. Overall, cortical regions mostly track the syllable rate, whereas subcortical regions track the acoustic envelope. Furthermore, with less intelligible speech, tracking of the modulation rate becomes more dominant. Our study highlights the importance of distinguishing between envelope modulation and syllable rate and provides novel possibilities to better understand differences between auditory processing and speech/language processing disorders. O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=160 SRC="FIGDIR/small/437033v1_ufig1.gif" ALT="Figure 1"> View larger version (23K): org.highwire.dtl.DTLVardef@6364adorg.highwire.dtl.DTLVardef@166daaeorg.highwire.dtl.DTLVardef@19eedd2org.highwire.dtl.DTLVardef@10f7aed_HPS_FORMAT_FIGEXP M_FIG C_FIG

neuroscience↗

bioRxiv2019

Phantom auditory perception (tinnitus) is characterised by stronger anticipatory auditory predictions

How phantom perceptions arise and the factors that make individuals prone to such experiences are not well understood. An attractive phenomenon to study these questions is tinnitus, a very common auditory phantom perception which is not explained by hyperactivity in the auditory pathway alone. Our framework posits that a predisposition to developing (chronic) tinnitus is dependent on individual traits relating to the formation and utilization of sensory predictions. Predictions of auditory stimulus frequency (remote from tinnitus frequency) were studied using a paradigm parametrically modulating regularity (i.e. predictability) of tone sequences and applying decoding techniques on magnetoencephalographic (MEG) data. For processes likely linked to short-term memory, individuals with tinnitus showed an enhanced anticipatory prediction pattern associated with increasing sequence regularity. In contrast, individuals without tinnitus engaged the same processes following the onset of the to-be-decoded sound. We posit that this tendency to optimally anticipate static and changing auditory inputs may determine which individuals faced with persistent auditory pathway hyperactivity factor it into auditory predictions, and thus perceive it as tinnitus. While our study constitutes a first step relating vulnerability to tinnitus with predictive processing, longitudinal studies are needed to confirm the predisposition model of tinnitus development.

neuroscience↗

bioRxiv2019

Degraded auditory and visual speech affects theta synchronization and alpha power differently

Understanding degraded speech, e.g. following a hearing damage, can pose a challenge. Previous attempts to quantify speech intelligibility in neural terms have usually focused on one of two measures, namely low-frequency speech-brain synchronization or alpha power modulations. However, reports have been mixed concerning the modulation of these measures, an issue aggravated by the fact that they have normally been studied separately. Using a parametric speech degradation approach, we present MEG studies that overcome this shortcoming. In a first study, participants listened to unimodal auditory speech with three different levels of degradation (original, 7-channel and 3-channel vocoding). Intelligibility declined with declining clarity, implemented by fewer vocoding channels but was still intelligible to some extent even for the lowest clarity level used (3-channel vocoding). Low- frequency (1-7 Hz) speech tracking suggested a u-shaped relationship with strongest effects for the medium degraded speech (7-channel) in bilateral auditory and left frontal regions. To follow up on this finding, we implemented three additional vocoding levels (5-channel, 2- channel, 1-channel) in a second MEG study. Using this wider range of degradation, the speech-brain synchronization showed a similar pattern for the degradation levels used in the first study but further shows that when speech becomes unintelligible, synchronization declines again. The relationship differed for alpha power, which continued to decrease across vocoding levels reaching a floor effect for 5-channel vocoding. Predicting subjective intelligibility based on models either combining both measures or each measure alone, showed superiority of the combined model. Our findings underline that speech tracking and alpha power are modified differently by the degree of degradation of continuous speech but together contribute to the subjective understanding of speech.

neuroscience↗