Biology subjects

Haider, C. L.

Publications and source records attributed to Haider, C. L..

4 recordsLinked to original sources

bioRxiv2024

Decreasing hearing ability does not lead to improved visual speech extraction as revealed in a neural speech tracking paradigm

The use of visual speech is thought to be especially important in situations where acoustics are unclear and in individuals with hearing impairment. To investigate this in a neural speech tracking paradigm, we measured MEG in sixty-seven mid- to old-age individuals during audiovisual (AV), audio-only (A), and visual-only (V) speech in the context of face masks. First, we could extend previous findings by showing that not only in young normal-hearing individuals but also in aging individuals with decreasing hearing ability the brain is superior in neurally tracking the acoustic spectrogram in AV compared to A presentations, especially in multi-speaker situations. The addition of visual lip movements further increases this benefit. Second, we could show that neural speech tracking in individuals with lower levels of hearing ability is affected more by face masks. However, in this population, the effect seems to be a composite of blocked visual speech and distorted acoustics. Third, we could confirm previous findings, that the neural benefit of visual speech varies strongly across individuals. We show that this general individual ability predicts how much people engage in visual speech tracking in difficult AV listening situations. Interestingly, this was not correlated with hearing thresholds and therefore does seem to be a widely used compensatory strategy in the hearing impaired.

neuroscience↗

bioRxiv2023

Lip movements and lexical features improve speech tracking differently for clear and multi-speaker speech

Visual speech plays a powerful role in facilitating auditory speech processing and has been a publicly noticed topic with the wide usage of face masks during the Covid-19 pandemic. In a previous magnetoencephalography (MEG) study we showed that occluding the mouth area significantly impairs neural speech tracking. To rule out the possibility that this deterioration is due to degraded sound quality, in the present follow-up study, we presented participants with audiovisual (AV) and audio-only (A) speech. We further independently manipulated the trials by adding a face mask and a distractor speaker. Our results clearly show that face masks only affect speech tracking in AV conditions, not in A conditions. This shows that face masks indeed primarily impact speech processing by blocking visual speech and not by acoustic degradation. Furthermore, we observe differences in the speech features that are used for visual speech processing. On the one hand, processing in clear speech, but not in noisy speech, is profiting more from lexical unit features (phonemes and word onsets) hinting at improved phoneme discrimination. On the other hand, we observe an improvement in speech tracking driven by the modulations of the lip area in clear speech and conditions with a distractor speaker, which might aid by providing temporal cues for subsequent auditory processing. With this work, we highlight the effects of face masks in AV speech tracking and show two separate ways how visual speech might support successful speech processing.

neuroscience↗

bioRxiv2023

Neural speech tracking benefit of lip movements predicts behavioral deterioration when the speaker's mouth is occluded

Observing lip movements of a speaker is known to facilitate speech understanding, especially in challenging listening situations. Converging evidence from neuroscientific studies shows enhanced processing of audiovisual stimuli. However, the interindividual variability of this visual benefit and its consequences on behavior are unknown. Here, we analyzed source-localized magnetoencephalographic (MEG) responses from normal-hearing participants listening to audiovisual speech with or without an additional distractor speaker. Using temporal response functions (TRFs), we show that neural responses to lip movements are, in general, enhanced when speech is challenging. After conducting a crucial control for speech acoustics, we show that lip movements effectively contribute to higher neural speech tracking, particularly when a distractor speaker is present. However, the extent of this visual benefit varied greatly among participants. Probing the behavioral relevance, we show that individuals who benefit more from lip movement information in terms of neural speech tracking, show a stronger drop in performance and an increase in perceived difficulty when the mouth is occluded by a surgical face mask. By contrast, no effect was found when the mouth was not occluded. We provide novel insights on how the benefit of lip movements in terms of neural speech tracking varies among individuals. Furthermore, we reveal its behavioral relevance by demonstrating negative consequences for behavior when visual speech is absent. Our results also offer potential implications for future objective assessments of audiovisual speech perception.

neuroscience↗

bioRxiv2021

Face masks impair reconstruction of acoustic speech features and higher-level segmentational features in the presence of a distractor speaker

Multisensory integration enables stimulus representation even when the sensory input in a single modality is weak. In the context of speech, when confronted with a degraded acoustic signal, congruent visual inputs promote comprehension. When this input is occluded speech comprehension consequently becomes more difficult. But it still remains inconclusive which levels of speech processing are affected under which circumstances by occlusion of the mouth area. To answer this question, we conducted an audiovisual (AV) multi-speaker experiment using naturalistic speech. In half of the trials, the target speaker wore a (surgical) face mask, while we measured the brain activity of normal hearing participants via magnetoencephalography (MEG). We additionally added a distractor speaker in half of the trials in order to create an ecologic difficult listening situation. A decoding model on the clear AV speech was trained and used to reconstruct crucial speech features in each condition. We found significant main effects of face masks on the reconstruction of acoustic features, such as the speech envelope and spectral speech features (i.e. pitch and formant frequencies), while reconstruction of higher level features of speech segmentation (phoneme and word onsets) were especially impaired through masks in difficult listening situations. As we used surgical face masks in our study, which only show mild effects on speech acoustics, we interpret our findings as the result of the occluded lip movements. This idea is in line with recent research showing that visual cortical regions track spectral modulations. Our findings extend previous behavioural results, by demonstrating the complex contextual effects of occluding relevant visual information on speech processing. HighlightsO_LISurgical face masks impair neural tracking of speech features C_LIO_LITracking of acoustic features is generally impaired, while higher level segmentational features show their effects especially in challenging listening situations C_LIO_LIAn explanation is the prevention of a visuo-phonological transformation contributing to audiovisual multisensory integration C_LI

neuroscience↗