Search bioRxiv⌕ Search

Biology subjects

Francart, T.

Publications and source records attributed to Francart, T..

At least 55 records · Page 3Linked to original sources

Fast EEG-based decoding of the directional focus of auditory attention using common spatial patterns

ObjectiveNoise reduction algorithms in current hearing devices lack information about the sound source a user attends to when multiple sources are present. To resolve this issue, they can be complemented with auditory attention decoding (AAD) algorithms, which decode the attention using electroencephalography (EEG) sensors. State-of-the-art AAD algorithms employ a stimulus reconstruction approach, in which the envelope of the attended source is reconstructed from the EEG and correlated with the envelopes of the individual sources. This approach, however, performs poorly on short signal segments, while longer segments yield impractically long detection delays when the user switches attention. MethodsWe propose decoding the directional focus of attention using filterbank common spatial pattern filters (FB-CSP) as an alternative AAD paradigm, which does not require access to the clean source envelopes. ResultsThe proposed FB-CSP approach outperforms both the stimulus reconstruction approach on short signal segments, as well as a convolutional neural network approach on the same task. We achieve a high accuracy (80% for 1 s windows and 70% for quasi-instantaneous decisions), which is sufficient to reach minimal expected switch durations below 4 s. We also demonstrate that the decoder can adapt to unlabeled data from an unseen subject and works with only a subset of EEG channels located around the ear to emulate a wearable EEG setup. ConclusionThe proposed FB-CSP method provides fast and accurate decoding of the directional focus of auditory attention. SignificanceThe high accuracy on very short data segments is a major step forward towards practical neuro-steered hearing devices.

neuroscience↗

EEG-informed speaker extraction from noisy recordings in neuro-steered hearing aids: linear versus deep learning methods

ObjectiveA hearing aids noise reduction algorithm cannot infer to which speaker the user intends to listen to. Auditory attention decoding (AAD) algorithms allow to infer this information from neural signals, which leads to the concept of neuro-steered hearing aids. We aim to evaluate and demonstrate the feasibility of AAD-supported speech enhancement in challenging noisy conditions based on electroencephalography (EEG) recordings. ApproachThe AAD performance with a linear versus a deep neural network (DNN) based speaker separation was evaluated for same-gender speaker mixtures using 3 different speaker positions and 3 different noise conditions. Main resultsAAD results based on the linear approach were found to be at least on par and sometimes even better than pure DNN-based approaches in terms of AAD accuracy in all tested conditions. However, when using the DNN to support a linear data-driven beamformer, a performance improvement over the purely linear approach was obtained in the most challenging scenarios. The use of multiple microphones was also found to improve speaker separation and AAD performance over single-microphone systems. SignificanceRecent proof-of-concept studies in this context each focus on a different method in a different experimental setting, which makes it hard to compare them. Furthermore, they are tested in highly idealized experimental conditions, which are still far from a realistic hearing aid setting. This work provides a systematic comparison of a linear and non-linear neuro-steered speech enhancement model, as well as a more realistic validation in challenging conditions.

bioengineering↗

From Modulated Noise to Natural Speech: the Effect of Stimulus Parameters on the Frequency Following Response

Frequency following responses (FFRs) can be evoked by a wide range of auditory stimuli, but for many stimulus parameters the effect on FFR strength is not fully understood. This complicates the comparison of earlier studies and the design of new studies. Furthermore, the most optimal stimulus parameters are unknown. To help resolve this issue, we investigated the effects of four important stimulus parameters and their interactions on the FFR. FFRs were measured in 16 normal hearing subjects evoked by stimuli with four levels of stimulus complexity (amplitude modulated noise, artificial vowels, natural vowels and nonsense words), three frequencies (around 105 Hz, 185 Hz and 245 Hz), three frequency contours (upward sweeping, downward sweeping and flat) and three vowels (Flemish /a:/, /u:/, and /i:/). We found that FFRs evoked by artificial vowels were on average 4 to 6 dB SNR larger than responses evoked by the other stimulus complexities, probably because of (unnaturally) strong higher harmonics. Moreover, response amplitude decreased with stimulus frequency but response SNR did not. Thirdly, frequency variation within the stimulus did not impact FFR strength, but only when rate of change remained low (e.g. not the case for sweeping natural vowels). Finally, the vowel /i:/ appeared to evoke larger response amplitudes compared to /a:/ and /u:/, but analysis power was too small to confirm this statistically. Differences in response strength between evoking vowels have been suggested to stem from destructive interference between response components. We show how a model of the auditory periphery can simulate these interference patterns and predict response strength. Altogether, the results of this study can guide stimulus choice for future FFR research and practical applications.

neuroscience↗

Top-down modulation of neural envelope tracking: the interplay with behavioral, self-reported and neural measures of listening effort

When listening to natural speech, our neural activity tracks the speech envelope. Moreover, recent research has demonstrated that this neural envelope tracking can be affected by top-down processes. The present study was designed to examine if neural envelope tracking is modulated by the effort that a person expends during listening. Five measures were included to quantify listening effort: two behavioral measures based on a novel dual-task paradigm, a self-report effort measure and two neural measures related to neural phase synchronization and alpha power. Electroencephalography responses to sentences, presented at a wide range of subject-specific signal-to-noise ratios, were recorded in thirteen young, normal-hearing adults. A comparison of the five measures revealed different effects of listening effort as a function of speech understanding. Reaction times on the primary task and self-reported effort decreased with increasing speech understanding. In contrast, reaction times on the secondary task and alpha power showed a peak-shaped behavior with highest effort at intermediate speech understanding levels. We found a positive association between envelope tracking and speech understanding. While a significant effect of listening effort was found on theta-band envelope tracking, the effect size was negligible. Therefore, our results suggest that listening effort is not a confound when using envelope tracking to objectively measure speech understanding in young, normal-hearing adults.

neuroscience↗

Hearing impairment is associated with enhanced neural tracking of the speech envelope

Elevated hearing thresholds in hearing impaired adults are usually compensated by providing amplification through a hearing aid. In spite of restoring hearing sensitivity, difficulties with understanding speech in noisy environments often remain. One main reason is that sensorineural hearing loss not only causes loss of audibility but also other deficits, including peripheral distortion but also central temporal processing deficits. To investigate the neural consequences of hearing impairment in the brain underlying speech-in-noise difficulties, we compared EEG responses to natural speech of 14 hearing impaired adults with those of 14 age-matched normal-hearing adults. We measured neural envelope tracking to sentences and a story masked by different levels of a stationary noise or competing talker. Despite their sensorineural hearing loss, hearing impaired adults showed higher neural envelope tracking of the target than the competing talker, similar to their normal-hearing peers. Furthermore, hearing impairment was related to an additional increase in neural envelope tracking of the target talker, suggesting that hearing impaired adults may have an enhanced sensitivity to envelope modulations or require a larger differential tracking of target versus competing talker to neurally segregate speech from noise. Lastly, both normal-hearing and hearing impaired participants showed an increase in neural envelope tracking with increasing speech understanding. Hence, our results open avenues towards new clinical applications, such as neuro-steered prostheses as well as objective and automatic measurements of speech understanding performance. HighlightsO_LIAdults with hearing impairment can neurally segregate speech from background noise C_LIO_LIHearing loss is related to enhanced neural envelope tracking of the target talker C_LIO_LINeural envelope tracking has potential to objectively measure speech understanding C_LI

neuroscience↗

The interplay of top-down focal attention and the cortical tracking of speech

AO_SCPLOWBSTRACTC_SCPLOWMany active neuroimaging paradigms rely on the assumption that the participant sustains attention to a task. However, in practice, there will be momentary distractions, potentially influencing the results. We investigated the effect of focal attention, objectively quantified using a measure of brain signal entropy, on cortical tracking of the speech envelope. The latter is a measure of neural processing of naturalistic speech. We let participants listen to 44 minutes of natural speech, while their electroencephalogram was recorded, and quantified both entropy and cortical envelope tracking. Focal attention affected the later brain responses to speech, between 100 and 300 ms latency. By only taking into account periods with higher attention, the measured cortical speech tracking improved by 47%. This illustrates the impact of the participants active engagement in the modeling of the brain-speech response and the importance of accounting for it. Our results suggests a cortico-cortical loop that initiates during the early-stages of the auditory processing, then propagates through the parieto-occipital and frontal areas, and finally impacts the later-latency auditory processes in a top-down fashion. The proposed framework could be transposed to other active electrophysiological paradigms (visual, somatosensory, etc) and help to control the impact of participants engagement on the results.

neuroscience↗

Optimal number and placement of EEG electrodes for measurement of neural tracking of speech

Measurement of neural tracking of natural running speech from the electroencephalogram (EEG) is an increasingly popular method in auditory neuroscience and has applications in audiology. The method involves decoding the envelope of the speech signal from the EEG signal, and calculating the correlation with the envelope of the audio stream that was presented to the subject. Typically EEG systems with 64 or more electrodes are used. However, in practical applications, set-ups with fewer electrodes are required. Here, we determine the optimal number of electrodes, and the best position to place a limited number of electrodes on the scalp. We propose a channel selection strategy based on an utility metric, which allows a quick quantitative assessment of the influence of a channel (or a group of channels) on the reconstruction error. We consider two use cases: a subject-specific case, where the optimal number and position of the electrodes is determined for each subject individually, and a subject-independent case, where the electrodes are placed at the same positions (in the 10-20 system) for all the subjects. We evaluated our approach using 64-channel EEG data from 90 subjects. In the subject-specific case we found that the correlation between actual and reconstructed envelope first increased with decreasing number of electrodes, with an optimum at around 20 electrodes, yielding 29% higher correlations using the optimal number of electrodes compared to all electrodes. This means that our strategy of removing electrodes can be used to improve the correlation metric in high-density EEG recordings. In the subject-independent case, we obtained a stable decoding performance when decreasing from 64 to 22 channels. When the number of channels was further decreased, the correlation decreased. For a maximal decrease in correlation of 10%, 32 well-placed electrodes were sufficient in 91% of the subjects.

neuroscience↗

An Interpretable Performance Metric for Auditory Attention Decoding Algorithms in a Context of Neuro-Steered Gain Control

In a multi-speaker scenario, a hearing aid lacks information on which speaker the user intends to attend, and therefore it often mistakenly treats the latter as noise while enhancing an interfering speaker. Recently, it has been shown that it is possible to decode the attended speaker from the brain activity, e.g., recorded by electroencephalography sensors. While numerous of these auditory attention decoding (AAD) algorithms appeared in the literature, their performance is generally evaluated in a non-uniform manner. Furthermore, AAD algorithms typically introduce a trade-off between the AAD accuracy and the time needed to make an AAD decision, which hampers an objective benchmarking as it remains unclear which point in each algorithms trade-off space is the optimal one in a context of neuro-steered gain control. To this end, we present an interpretable performance metric to evaluate AAD algorithms, based on an adaptive gain control system, steered by AAD decisions. Such a system can be modeled as a Markov chain, from which the minimal expected switch duration (MESD) can be calculated and interpreted as the expected time required to switch the operation of the hearing aid after an attention switch of the user, thereby resolving the trade-off between AAD accuracy and decision time. Furthermore, we show that the MESD calculation provides an automatic and theoretically founded procedure to optimize the number of gain levels and decision time in an AAD-based adaptive gain control system.

neuroscience↗

Effect of task and attention on neural tracking of speech

EEG-based measures of neural tracking of natural running speech are becoming increasingly popular to investigate neural processing of speech and have applications in audiology. When the stimulus is a single speaker, it is usually assumed that the listener actively attends to and understands the stimulus. However, as the level of attention of the listener is inherently variable, we investigated how it affected neural envelope tracking. Using a movie as a distractor, we varied the level of attention while we estimated neural envelope tracking. We varied the intelligibility level by adding stationary noise. We found a significant difference in neural envelope tracking between the condition with maximal attention and the movie condition. This difference was most pronounced in the right-frontal region of the brain. The degree of neural envelope tracking was highly correlated with the stimulus signal-to-noise ratio, even in the movie condition. This could be due to residual neural resources to passively attend to the stimulus. When envelope tracking is used to objectively measure speech understanding this means that the procedure can be made more enjoyable and feasible by letting participants watch a movie during stimulus presentation.

neuroscience↗

Data-driven spatial filtering for improved measurement of cortical tracking of multiple representations of speech

ObjectiveMeasurement of the cortical tracking of continuous natural speech from electroencephalography (EEG) recordings using a forward model is becoming an important tool in auditory neuroscience. However, it requires a manual channel selection based on visual inspection or prior knowledge to obtain a summary measure of cortical tracking. In this study, we present a method to on the one hand remove non-stimulus-related activity from the EEG signals to be predicted, and on the other hand automatically select the channels of interest. We also aim to show that the EEG prediction from phonology-related speech features is possible in Dutch. ApproachEighteen participants listened to a Flemish story, while their EEG was recorded. Subject-specific and grand-average temporal responses functions were determined between the EEG activity in different frequency bands and several stimulus features: the envelope, spectrogram, phonemes, phonetic features or a combination. The temporal response functions were then used to predict EEG from the stimulus, and the predicted was compared with the recorded EEG, yielding a measure of cortical tracking of stimulus features. A spatial filter was calculated based on the generalized eigenvalue decomposition (GEVD), and the effect on EEG prediction accuracy was determined. Main resultsA model including both low- and high-level speech representations was able to better predict the brain responses to the speech than a model only including low-level features. The inclusion of a GEVD-based spatial filter in the model increased the prediction accuracy of cortical responses to each speech feature at both single-subject (270% improvement) and group-level (310 %). SignificanceWe showed that the inclusion of acoustical and phonetic speech information and the addition of a data-driven spatial filter allow improved modelling of the relationship between the speech and its brain response and offer an automatic channel selection. HighlightsO_LIAutomatic channel selection for evaluating the cortical tracking of continuous natural speech C_LIO_LIData-driven spatial filtering for removing non-stimulus-related activity from the EEG signals C_LIO_LIImproved prediction of brain responses to speech by combining acoustical and phonetic speech information in Dutch C_LI DisclosureThe authors report no disclosures relevant to the manuscript.

neuroscience↗

Stimulus-aware spatial filtering for single-trial neural response and temporal response function estimation in high-density EEG with applications in auditory research

Objective. Neural responses recorded using electroencephalography (EEG) and magnetoencephalography (MEG) can be used to study how our brain functions, as well as for various promising brain computer interface (BCI) applications. However, a common problem is the low signal to noise ratio (SNR) which makes it challenging to estimate task-related neural responses or the temporal response function (TRF) describing the linear relationship between the stimulus and the neural response, particularly over short data windows. To address these, we present an algorithm that takes advantage of the multi-channel nature of the recordings, and knowledge of the presented stimulus, to achieve a joint noise reduction and dimensionality reduction using spatial filtering. Methods. Forward modeling is used to project the stimulus onto the electrode space. The second-order statistics of this estimated desired signal and the raw neural data are used to estimate spatial filters that maximize the SNR of the neural response, based on a generalized eigenvalue decomposition. Main Results. 1. For synthesized EEG data, over a range of SNRs, our filtering resulted in significantly better TRF estimates from 20 s trials, compared to unfiltered EEG data. 2. On a dataset from 28 subjects who listened to a single-talker stimulus, our method resulted in correlations between predicted neural responses and the original EEG data that were significantly higher compared to standard approaches. 3. On a dataset of 16 subjects attending to 1 speaker in a two-speaker scenario, our method resulted in attention decoding accuracies which were higher compared to existing forward modelling methods. Significance. Our algorithm presents a data-driven way to denoise and reduce dimensionality of neural data, thus aiding further analysis, by utilizing the knowledge of the stimulus. The method is computationally efficient, and does not require repeated trials, thereby relieving experiment design from the necessity of presenting repeated stimuli to the subjects.

neuroscience↗

The effect of stimulus envelope shape on the auditory steady-state response

Auditory steady-state responses (ASSRs) are auditory evoked potentials that reflect phase-locked neural activity to periodic stimuli. ASSRs are often evoked by tones with a modulated envelope, with sinusoidal envelopes being most common. However, it is unclear if and how the shape of the envelope affects ASSR responses. In this study, we used various trapezoidal modulated tones to evoke ASSRs (modulation frequency = 40 Hz) and studied the effect of four envelope parameters: attack time, hold time, decay time and off time. ASSR measurements in 20 normal hearing subjects showed that envelope shape significantly influenced responses: increased off time and/or increased decay time led to responses with a larger signal-to-noise-ratio (SNR). Response phase delay was significantly influenced by attack time and to a lesser degree by off time. We also simulated neural population responses that approximate ASSRs with a model of the auditory periphery (Bruce et al. 2018). The modulation depth of the simulated responses, i.e. the difference between maximum and minimum firing rate, correlated highly with the response SNRs found in the ASSR measurements. Longer decay time and off time enhanced the modulation depth both by decreasing the minimum firing rate and by increasing the maximum firing rate. In conclusion, custom envelopes with long decay and off time provide larger response SNRs and the benefit over the commonly used sinusoidal envelope was in the range of several dB.

neuroscience↗

Evidence for enhanced neural tracking of the speech envelope underlying age-related speech-in-noise difficulties

Elevated hearing thresholds in hearing impaired adults are usually compensated by providing amplification through a hearing aid. In spite of restoring hearing sensitivity, difficulties with understanding speech in noisy environments often remain. One main reason is that sensorineural hearing loss not only causes loss of audibility but also other deficits, including peripheral distortion but also central temporal processing deficits. To investigate the neural consequences of hearing impairment in the brain underlying speech-in-noise difficulties, we compared EEG responses to natural speech of 14 hearing impaired adults with those of 14 age-matched normal-hearing adults. We measured neural envelope tracking to sentences and a story masked by different levels of a stationary noise or competing talker. Despite their sensorineural hearing loss, hearing impaired adults showed higher neural envelope tracking of the target than the competing talker, similar to their normal-hearing peers. Furthermore, hearing impairment was related to an additional increase in neural envelope tracking of the target talker, suggesting that hearing impaired adults may have an enhanced sensitivity to envelope modulations or require a larger differential tracking of target versus competing talker to neurally segregate speech from noise. Lastly, both normal-hearing and hearing impaired participants showed an increase in neural envelope tracking with increasing speech understanding. Hence, our results open avenues towards new clinical applications, such as neuro-steered prostheses as well as objective and automatic measurements of speech understanding performance. HighlightsO_LIAdults with hearing impairment can neurally segregate speech from background noise C_LIO_LIHearing loss is related to enhanced neural envelope tracking of the target talker C_LIO_LINeural envelope tracking has potential to objectively measure speech understanding C_LI

neuroscience↗

EEG-based detection of the attended speaker and the locus of auditory attention with convolutional neural networks

In a multi-speaker scenario, the human auditory system is able to attend to one particular speaker of interest and ignore the others. It has been demonstrated that it is possible to use electroencephalography (EEG) signals to infer to which speaker someone is attending by relating the neural activity to the speech signals. However, classifying auditory attention within a short time interval remains the main challenge. We present a convolutional neural network-based approach to extract the locus of auditory attention (left/right) without knowledge of the speech envelopes. Our results show that it is possible to decode the locus of attention within 1 to 2 s, with a median accuracy of around 81%. These results are promising for neuro-steered noise suppression in hearing aids, in particular in scenarios where per-speaker envelopes are unavailable.

neuroscience↗

Predicting individual speech intelligibility from the neural tracking of acoustic- and phonetic-level speech representations

ObjectiveTo objectively measure speech intelligibility of individual subjects from the EEG, based on cortical tracking of different representations of speech: low-level acoustical, higher-level discrete, or a combination. To compare each models prediction of the speech reception threshold (SRT) for each individual with the behaviorally measured SRT.\n\nMethodsNineteen participants listened to Flemish Matrix sentences presented at different signal-to-noise ratios (SNRs), corresponding to different levels of speech understanding. For different EEG frequency bands (delta, theta, alpha, beta or low-gamma), a model was built to predict the EEG signal from various speech representations: envelope, spectrogram, phonemes, phonetic features or a combination of phonetic Features and Spectrogram (FS). The same model was used for all subjects. The model predictions were then compared to the actual EEG of each subject for the different SNRs, and the prediction accuracy in function of SNR was used to predict the SRT.\n\nResultsThe model based on the FS speech representation and the theta EEG band yielded the best SRT predictions, with a difference between the behavioral and objective SRT below 1 decibel for 53% and below 2 decibels for 89% of the subjects.\n\nConclusionA model including low- and higher-level speech features allows to predict the speech reception threshold from the EEG of people listening to natural speech. It has potential applications in diagnostics of the auditory system.\n\nSearch Termscortical speech tracking, objective measure, speech intelligibility, auditory processing, speech representations.\n\nHighlightsO_LIObjective EEG-based measure of speech intelligibility\nC_LIO_LIImproved prediction of speech intelligibility by combining speech representations\nC_LIO_LICortical tracking of speech in the delta EEG band monotonically increased with SNRs\nC_LIO_LICortical responses in the theta EEG band best predicted the speech reception threshold\nC_LI\n\nDisclosureThe authors report no disclosures relevant to the manuscript.

neuroscience↗

Neural envelope tracking as a measure of speech understanding in cochlear implant users

The speech envelope is essential for speech understanding and can be reconstructed from the electroencephalogram (EEG) recorded while listening to running speech. This so-called neural envelope tracking has been shown to relate to speech understanding in normal hearing listeners, but has barely been investigated in persons wearing cochlear implants (CI). We investigated the relation between speech understanding and neural envelope tracking in CI users.\n\nEEG was recorded in 8 CI users while they listened to a story. Speech understanding was varied by changing the intensity of the presented speech. The speech envelope was reconstructed from the EEG using a linear decoder and then correlated with the envelope of the speech stimulus as a measure of neural envelope tracking which was compared to actual speech understanding.\n\nThis study showed that neural envelope tracking increased with increasing speech understanding in every participant. Furthermore behaviorally measured speech understanding was correlated with participant specific neural envelope tracking results indicating the potential of neural envelope tracking as an objective measure of speech understanding in CI users. This could enable objective and automatic fitting of CIs and pave the way towards closed-loop CIs that adjust continuously and automatically to individual CI users.

neuroscience↗

The effect of presentation level on spectrotemporal modulation detection

The understanding of speech in noise relies (at least partially) on spectrotemporal modulation sensitivity. This sensitivity can be measured by spectral ripple tests, which can be administered at different presentation levels. However, it is not known how presentation level affects spectrotemporal modulation thresholds. In this work, we present behavioral data for normal-hearing adults which show that at higher ripple densities (2 and 4 ripples/oct), increasing presentation level led to worse discrimination thresholds. Results of a computational model suggested that the higher thresholds could be explained by a worsening of the spectrotemporal representation in the auditory nerve due to broadening of cochlear filters and neural activity saturation. Our results demonstrate the importance of taking presentation level into account when administering spectrotemporal modulation detection tests.

neuroscience↗

Semantic context enhances neural envelope tracking

ObjectivesRecently an objective measure of speech intelligibility, based on brain responses derived from the electroencephalogram (EEG), has been developed using isolated Matrix sentences as a stimulus. We investigated whether this objective measure of speech intelligibility can also be used with natural speech as a stimulus, as this would be beneficial for clinical applications. DesignWe recorded the EEG in 19 normal-hearing participants while they listened to two types of stimuli: Matrix sentences and a natural story. Each stimulus was presented at different levels of speech intelligibility by adding speech weighted noise. Speech intelligibility was assessed in two ways for both stimuli: (1) behaviorally and (2) objectively by reconstructing the speech envelope from the EEG using a linear decoder and correlating it with the acoustic envelope. We also calculated temporal response functions (TRFs) to investigate the temporal characteristics of the brain responses in the EEG channels covering different brain areas. ResultsFor both stimulus types the correlation between the speech envelope and the reconstructed envelope increased with increasing speech intelligibility. In addition, correlations were higher for the natural story than for the Matrix sentences. Similar to the linear decoder analysis, TRF amplitudes increased with increasing speech intelligibility for both stimuli. Remarkable is that although speech intelligibility remained unchanged in the no noise and +2.5 dB SNR condition, neural speech processing was affected by the addition of this small amount of noise: TRF amplitudes across the entire scalp decreased between 0 to 150 ms, while amplitudes between 150 to 200 ms increased in the presence of noise. TRF latency changes in function of speech intelligibility appeared to be stimulus specific: The latency of the prominent negative peak in the early responses (50-300 ms) increased with increasing speech intelligibility for the Matrix sentences, but remained unchanged for the natural story. ConclusionsThese results show (1) the feasibility of natural speech as a stimulus for the objective measure of speech intelligibility, (2) that neural tracking of speech is enhanced using a natural story compared to Matrix sentences and (3) that noise and the stimulus type can change the temporal characteristics of the brain responses. These results might reflect the integration of incoming acoustic features and top-down information, suggesting that the choice of the stimulus has to be considered based on the intended purpose of the measurement.

neuroscience↗