Biology subjects

Accou, B.

Publications and source records attributed to Accou, B..

6 recordsLinked to original sources

bioRxiv2026

Interpersonal Synchronization of Brain and Body Tracks Attention and Listening Engagement

Purpose: Speech perception is a fundamental part of everyday communication that relies on more than simple identification of words and sentences. Attention and listening engagement both contribute to speech perception, while representing distinct aspects of the listening experience. Attention is typically associated with cognitive focus, whereas listening engagement additionally involves cognitive and affective immersion in sound. Despite their importance, these states remain difficult to disentangle, behaviorally and physiologically. Both have been linked to interpersonal synchronization (the synchronization of biobehavioral signals across individuals), raising questions about what this synchronization actually reflects. Method: In this study, we disentangled attention and listening engagement by independently manipulating both factors within a single experiment. Thirty participants listened to two simultaneously presented streams of meaningful speech and were instructed to focus on only one. Both attended and unattended stimuli were designed to be either engaging or non-engaging. Neural activity was recorded using EEG, while physiological responses were measured using heart rate and electrodermal activity. Results: Interpersonal synchronization was computed from neural and bodily signals, alongside a self-report measure of listening engagement and auditory attention decoding (AAD), a neural measure of selective attention. Interpersonal synchronization of all three modalities significantly predicted listening engagement, whereas neural interpersonal synchronization was the only measure that significantly predicted attention. These findings suggest that attention is primarily driven by cognitive processes represented in the brain, while listening engagement additionally involves affective processes that are more strongly reflected in bodily responses. Conclusions: Overall, this study demonstrates that different forms of interpersonal synchronization reflect distinct dimensions of the listening experience and supports interpersonal synchronization as a potential objective marker of listening engagement.

neuroscience↗

bioRxiv2025

Interpersonal synchronization as an objective measure of listening engagement

Listening engagement plays a crucial role in effective communication and knowledge transfer, reflecting a state of deep absorption in an auditory stimulus. The available behavioral assessments of listening engagement are limited in capturing its multidimensional nature. Measuring listening engagement objectively using brain and body signals is a compelling alternative. In particular, interpersonal synchronization--observed through synchronized neural and physiological responses among listeners--may offer a promising objective measure. In this study, we examined synchronization of brain activity, heart rate, and electrodermal activity between participants while they listened to engaging and non-engaging stories. Subjective engagement ratings were collected as a ground truth reference. Results showed higher interpersonal synchronization across all modalities when participants listened to engaging stories compared to non-engaging ones, with significant correlations between synchronization measures and ratings of subjective engagement. These results confirm that synchronization may be a reliable, objective marker of listening engagement. We anticipate that these results will provide a powerful framework for future research in relevant domains, including auditory neuroscience, education, and communication science, and enable impactful real-world applications, such as automatically tracking audience engagement in education, entertainment, and broader societal contexts.

neuroscience↗

bioRxiv2024

Classifying native versus foreign speech perception from EEG using linguistic speech features

When a person listens to natural speech, the relation between features of the speech signal and the corresponding evoked electroencephalogram (EEG) is indicative of neural processing of the speech signal. Using linguistic representations of speech, we investigate the differences in neural processing between speech in a native and foreign language that is not understood. We conducted experiments using three stimuli: a comprehensible language, an incomprehensible language, and randomly shuffled words from a comprehensible language, while recording the EEG signal of native Dutch-speaking participants. We modeled the neural tracking of linguistic features of the speech signals using a deep-learning model in a match-mismatch task that relates EEG signals to speech, while accounting for lexical segmentation features reflecting acoustic processing. The deep learning model effectively classifies languages. We also observed significant differences in tracking patterns between comprehensible and incomprehensible speech stimuli within the same language. It demonstrates the potential of deep learning frameworks in measuring speech understanding objectively.

neuroscience↗

bioRxiv2023

SparrKULee: A Speech-evoked Auditory ResponseRepository of the KU Leuven, containing EEG of 85participants

Researchers investigating the neural mechanisms underlying speech perception often employ electroencephalography (EEG) to record brain activity while participants listen to spoken language. The high temporal resolution of EEG enables the study of neural responses to fast and dynamic speech signals. Previous studies have successfully extracted speech characteristics from EEG data and, conversely, predicted EEG activity from speech features. Machine learning techniques are generally employed to construct encoding and decoding models, which necessitate a substantial amount of data. We present SparrKULee: A Speech-evoked Auditory Repository of EEG, measured at KU Leuven, comprising 64-channel EEG recordings from 85 young individuals with normal hearing, each of whom listened to 90-150 minutes of natural speech. This dataset is more extensive than any currently available dataset in terms of both the number of participants and the amount of data per participant. It is suitable for training larger machine learning models. We evaluate the dataset using linear and state-of-the-art non-linear models in a speech encoding/decoding and match/mismatch paradigm, providing benchmark scores for future research.

neuroscience↗

bioRxiv2023

Robust neural tracking of linguistic speech representations using a convolutional neural network.

ObjectiveWhen listening to continuous speech, populations of neurons in the brain track different features of the signal. Neural tracking can be measured by relating the electroencephalography (EEG) and the speech signal. Recent studies have shown a significant contribution of linguistic features over acoustic neural tracking using linear models. However, linear models cannot model the nonlinear dynamics of the brain. To overcome this, we use a convolutional neural network (CNN) that relates EEG to linguistic features using phoneme or word onsets as a control and has the capacity to model non-linear relations. ApproachWe integrate phoneme- and word-based linguistic features (phoneme surprisal, cohort entropy, word surprisal and word frequency) in our nonlinear CNN model and investigate if they carry additional information on top of lexical features (phoneme and word onsets). We then compare the performance of our nonlinear CNN with that of a linear encoder and a linearized CNN. Main resultsFor the non-linear CNN, we found a significant contribution of cohort entropy over phoneme onsets and of word surprisal and word frequency over word onsets. Moreover, the non-linear CNN outperformed the linear baselines. SignificanceMeasuring coding of linguistic features in the brain is important for auditory neuroscience research and applications that involve objectively measuring speech understanding. With linear models, this is measurable, but the effects are very small. The proposed non-linear CNN model yields larger differences between linguistic and lexical models and, therefore, could show effects that would otherwise be unmeasurable and may, in the future, lead to improved within-subject measures and shorter recordings. Index TermsEEG decoding, speech processing, CNN, linguistics.

neuroscience↗

bioRxiv2022

Decoding of the speech envelope from EEG using the VLAAI deep neural network.

To investigate the processing of speech in the brain, commonly simple linear models are used to establish a relationship between brain signals and speech features. However, these linear models are ill-equipped to model a highly-dynamic, complex non-linear system like the brain, and they often require a substantial amount of subject-specific training data. This work introduces a novel speech decoder architecture: the Very Large Augmented Auditory Inference (VLAAI) network. The VLAAI network outperformed state-of-the-art subject-independent models (median Pearson correlation of 0.19, p < 0.001), yielding an increase over the well-established linear model by 52%. Using ablation techniques we identified the relative importance of each part of the VLAAI network and found that the non-linear components and output context module influenced model performance the most (10% relative performance increase). Subsequently, the VLAAI network was evaluated on a holdout dataset of 26 subjects and publicly available unseen dataset to test generalization for unseen subjects and stimuli. No significant difference was found between the holdout subjects and the default test set, and only a small difference between the default test set and the public dataset was found. Compared to the baseline models, the VLAAI network still significantly outperformed all baseline models on the public dataset. We evaluated the effect of training set size by training the VLAAI network on data from 1 up to 80 subjects and evaluated on 26 holdout subjects, revealing a logarithmic relationship between the number of subjects in the training set and the performance on unseen subjects. Finally, the subject-independent VLAAI network was fine-tuned for 26 holdout subjects to obtain subject-specific VLAAI models. With 5 minutes of data or more, a significant performance improvement was found, up to 34% (from 0.18 to 0.25 median Pearson correlation) with regards to the subject-independent VLAAI network.

neuroscience↗