Biology subjects

Di Liberto, G. M.

Publications and source records attributed to Di Liberto, G. M..

15 recordsLinked to original sources

bioRxiv2026

Visual speech supports phonetic attunement in the absence of early auditory input

The first year of life is considered a sensitive period for the acquisition of phonetic categories, a hallmark of successful native language specialization. The extent to which this process depends on early auditory experience and intrinsic biological constraints remains unresolved. We measured neural encoding of continuous natural speech in hearing children (HC) and cochlear implant (CI) users with congenital or acquired deafness, contrasting children with and without access to auditory input in the first year of life. Speech encoding was present across all groups, but its specificity depended on early input: auditory phonetic features were encoded only in children exposed to speech within the first year, whereas visually discriminable phonetic features were encoded regardless of auditory deprivation. These findings show that early sensory input gates phonetic attunement; this constraint is not limited to or grounded in audition but instead reveals a sensitive period that is modality-flexible in mechanism and experience-dependent in expression.

neuroscience↗

bioRxiv2026

Investigating neural speech processing with functional near infrared spectroscopy: considerations for temporal response functions

Functional near infrared spectroscopy (fNIRS) is increasingly used in hearing and communication research, with advantages such as robustness to movement artifacts, improved spatial resolution, and flexibility of contexts in which it can be applied. At the same time, the field is progressively moving towards more continuous, naturalistic listening paradigms resulting in the widespread adoption of speech tracking analyses such as temporal response functions (TRFs) in electroencephalography (EEG) and magnetoencephalography (MEG) studies. However, it remains unclear whether these analyses can be applied to slower haemodynamic signals measured by fNIRS. In the present study, we investigated whether a TRF framework can similarly be applied to fNIRS data recorded during continuous speech perception. Eight participants listened to speech simultaneously while fNIRS signals were acquired in a hyperscanning setup. Speech features were regressed onto the haemodynamic responses to test the feasibility and interpretability of fNIRS-based TRFs. Prediction correlations between observed and modelled fNIRS signals across speech features were higher than those typically reported for EEG- and comparable to those reported for MEG-TRF studies. Moreover, these correlations did not overlap with a null distribution generated from triallJmismatched fNIRS data, confirming statistical significance and were slightly greater than those obtained from a conventional GLM approach. Our findings support that TRF estimation method can yield meaningful and statistically significant responses from fNIRS data. HighlightsO_LITRF modelling can be meaningfully applied to fNIRS data acquired during speech listening tasks. C_LIO_LIPrediction correlations between actual and modelled fNIRS signals were above chance level, with values comparable to previous EEG/MEG studies. C_LIO_LITRFs explained more fNIRS variance than a conventional GLM approach. C_LI

neuroscience↗

bioRxiv2026

Trust Modulates Speech Entrainment: Enhanced Cortical Tracking for Low Trust Speakers

Trust is a critical component of human communication, providing a foundation for understanding, information exchange, and social coordination. Much of the research on trust in speech communication has focused on how vocal characteristics impact perceived trustworthiness. However, little is known about how trust in a speaker affects the neural processing of speech. Here, we demonstrate a two-stage experimental framework to study that question using non-invasive EEG. First, participants engage in a trust-building stage, where they play an investment game with fictional characters, each paired with a distinctive voice and trustworthiness level (i.e., frequency and magnitude of lies). Next, participants engage in a story-listening stage, in which they are presented with stories from the same characters. Data acquired from twenty young adults confirm a statistically significant correlation between the perceived and actual trustworthiness of the fictional characters. Cortical speech tracking was quantified using a temporal response function (TRF) analysis on the EEG data. We found that the trustworthiness established during the trust-building stage influenced the cortical tracking of speech in the subsequent story-listening stage, with lower trustworthiness corresponding to a stronger cortical tracking of speech. Interestingly, trustworthiness selectively modulated tracking strength, with no statistically significant changes in how language is represented across space and time.

neuroscience↗

bioRxiv2025

Exploring the impact of social relevance on the cortical tracking of speech: viability and temporal response characterisation

Human speech is inherently social. Yet our understanding of the neural substrates underlying continuous speech perception relies largely on neural responses to monologues, leaving substantial uncertainty about how social interactions shape the neural encoding of speech. Here, we bridge this gap by studying how EEG responses to speech change when the input includes a social element. In Experiment 1, we compared the neural encoding of synthesised undirected monologues, directed monologues, and dialogues. In Experiment 2, we extended this by using podcasts, addressing the additional challenges of real speech dialogue, such as dysfluency. Using temporal response function analyses, we show that the presence of a social component strengthens the cortical tracking of the speech envelope, despite identical acoustic properties. Neural responses to synthesised speech showed a strong correlation with those for real speech podcasts, with a stronger alignment emerging for more socially-relevant speech material. In addition, we demonstrate that robust neural indices of sound and lexical-level processing can be derived using real podcast recordings despite the presence of dysfluencies. Finally, we present a simulation to put to the test the robustness of temporal response function analyses under increasing levels of dysfluency. Together, these findings highlighting the impact of social elements in shaping auditory neural processing, providing a framework for future investigation and analysis of social speech listening and speech interaction. Significance StatementHuman speech is rarely produced or processed in a social vacuum. Yet, our understanding of continuous speech neurophysiology mostly comes from experiments involving speech monologues. This study reveals how social context modulates the neural encoding of speech. We directly contrast neural signals recorded when participants listened to monologues and dialogues, using controlled material from speech synthesis and real podcast recordings. We found that the social element amplifies the neural encoding of speech features, reflecting greater engagement. We also show strong correlation between synthetic and real podcast neural responses, scaling with social relevance. Finally, we demonstrate that lexical processing can be measured robustly even amid natural dysfluencies. These insights advance our understanding of speech neurophysiology, informing future research on social speech.

neuroscience↗

bioRxiv2025

Where is the melody? Spontaneous attention orchestrates melody formation during polyphonic music listening

Humans seamlessly process multi-voice music into a coherent perceptual whole. Yet the neural strategies supporting this experience remain unclear. One fundamental component of this process is the formation of melody, a core structural element of music. Previous work on monophonic listening has provided strong evidence for the neurophysiological basis of melody processing, for example indicating predictive processing as a foundational mechanism underlying melody encoding. However, considerable uncertainty remains about how melodies are formed during polyphonic music listening, as existing theories (e.g., divided attention, figure-ground model, stream integration) fail to unify the full range of empirical findings. Here, we combined behavioural measures with non-invasive electroencephalography (EEG) to probe spontaneous attentional bias and melodic expectation while participants listened to two-voice classical excerpts. Our uninstructed listening paradigm eliminated a major experimental constraint, creating a more ecologically valid setting. We found that attention bias was significantly influenced by both the high-voice superiority effect and intrinsic melodic statistics. We then employed transformer-based models to generate next-note expectation profiles and test competing theories of polyphonic perception. Drawing on our findings, we propose a weighted-integration framework in which attentional bias calibrates the overall degree of integration of the competing streams. In doing so, the proposed framework reconciles previous divergent accounts by showing that, even under free-listening conditions, melodies emerge through an attention-guided statistical integration mechanism. HighlightsO_LIEEG can be used to decode spontaneous attention during the uninstructed listening of polyphonic music. C_LIO_LIBehavioural and neural data indicate that spontaneous attention is influenced by both high-voice superiority and melodic contour. C_LIO_LIAttention bias impacts the neural encoding of the polyphonic streams, with strongest effects within 200 ms after note onset. C_LIO_LIStimuli that produced a stronger attention bias aligned with monophonic-model expectations, whereas stimuli with a weaker bias aligned with the Stream-Integration model. C_LIO_LIWe propose a bi-directional influence between attention and prediction mechanisms, with horizontal statistics impacting attention (i.e., salience), and attention impacting melody extraction. C_LI

neuroscience↗

bioRxiv2025

Simultaneous cortical tracking of competing speech streams during attention switching

Successful speech communication in multi-talker scenarios requires a skilful combination of sustained attention and rapid attention switching. While the neurophysiology literature offers detailed insights into the neural underpinnings of sustained attention, there remains considerable uncertainty on how attention switching takes place. In this study, using EEG recordings from normal-hearing adults in an immersive multi-talker environment, we measured the neural encoding of two competing speech streams amid background babble. Participants were cued to switch attention between streams every 15-30 seconds. Neural tracking was assessed via Temporal Response Functions (TRF), confirming reliable decoding of attentional focus. Our results indicate asymmetric disengagement and engagement processes during attention switches, where the neural tracking of the new target stream emerges before disengaging from the previous target, revealing a transient simultaneous encoding of two speech streams. That transition was closely mirrored by a reduction in EEG alpha power, informing on the cognitive effort during different phases of the attention switch. We then isolated cortical activity reflecting lexical prediction mechanisms to determine how lexical context is updated after an attention switch, comparing four numerical hypotheses that were constructed using Large Language Models. Our findings elucidate both the temporal and contextual mechanisms underlying auditory attention shifts, pointing to the possibility that listeners carry out a reset in lexical context after switching attention. By focusing on dynamic attentional reallocation, this study offers insights into the brains capacity for flexible speech processing in complex listening environments.

neuroscience↗

bioRxiv2024

Are you talking to me? How the choice of speech register impacts listeners' hierarchical encoding of speech.

Speakers accommodate their speech to meet the needs of their listeners, producing different speech registers. One such register is Foreigner-Directed Speech (FDS), which is the way native speakers address non-native listeners, typically characterized by features such as slow speech rate and phonetic exaggeration. Here, we investigated how register impacts the cortical encoding of speech at different levels of language integration. Specifically, we tested the hypothesis that enhanced comprehension of FDS compared with Native-Directed Speech (NDS) involves more than just a slower speech rate, influencing speech processing from acoustic to semantic levels. Electroencephalography (EEG) signals were recorded from Spanish native listeners, who were learning English (L2 learners), and English native listeners (L1 listeners) as they were presented with audio-stories. Speech was presented in English in three different speech registers: FDS, NDS and a control register (Slow-NDS) which is slowed down version of NDS. We measured the cortical tracking of acoustic, phonological, and semantic information with a multivariate temporal response function analysis (TRF) on the EEG signals. We found that FDS promoted L2 learners cortical encoding at all the levels of speech and language processing considered. First, FDS led to a more pronounced encoding of the speech envelope. Second, phonological encoding was more refined when listening to FDS, with phoneme perception getting closer to that of L1 listeners. Finally, FDS also enhanced the TRF- N400, a neural signature of lexical expectations. Conversely FDS impacted acoustic but not linguistic speech encoding in L1 listeners. Taken together, these results support our hypothesis that FDS accommodates speech processing in L2 listeners beyond what can be achieved by simply speaking slowly, impacting the cortical encoding of sound and language at different abstraction levels. In turn, this study provides objective metrics that are sensitive to the impact of register on the hierarchical encoding of speech, which could be extended to other registers and cohorts.

neuroscience↗

bioRxiv2022

Infant low-frequency EEG cortical power, cortical tracking and phase-amplitude coupling predicts language a year later.

Cortical signals have been shown to track acoustic and linguistic properties of continuous speech. This phenomenon has been measured in both children and adults, reflecting speech understanding by adults as well as cognitive functions such as attention and prediction. Furthermore, atypical low-frequency cortical tracking of speech is found in children with phonological difficulties (developmental dyslexia). Accordingly, low-frequency cortical signals may play a critical role in language acquisition. A recent investigation with infants Attaheri et al., 2022 (1) probed cortical tracking mechanisms at the ages of 4, 7 and 11 months as participants listened to sung speech. Results from temporal response function (TRF), phase-amplitude coupling (PAC) and dynamic theta-delta power (PSD) analyses indicated speech envelope tracking and stimulus-related power (PSD) for delta and theta neural signals. Furthermore, delta- and theta-driven PAC was found at all ages, with theta phases displaying stronger PAC with high-frequency amplitudes than delta. The present study tests whether these previous findings replicate in the second half of the full cohort of infants (N = 122) who were participating in this longitudinal study (first half: N=61, (1); second half: N=61). In addition to demonstrating good replication, we investigate whether cortical tracking in the first year of life predicts later language acquisition for the full cohort (122 infants recruited, 113 retained) using both infant-led and parent-estimated measures and multivariate and univariate analyses. Increased delta cortical tracking in the univariate analyses, increased [~]2Hz PSD power and stronger theta-gamma PAC in both multivariate and univariate analyses were related to better language outcomes using both infant-led and parent-estimated measures. By contrast, increased [~]4Hz PSD power in the multi-variate analyses, increased delta-beta PAC and a higher theta/delta power ratio in the multi-variate analyses were related to worse language outcomes. The data are interpreted within a "Temporal Sampling" framework for developmental language trajectories.

neuroscience↗

bioRxiv2022

Atypical cortical encoding of speech identifies children with Dyslexia versus Developmental Language Disorder

Slow cortical oscillations play a crucial role in processing the speech envelope, which is perceived atypically by children with Developmental Language Disorder (DLD) and developmental dyslexia. Here we use electroencephalography (EEG) and natural speech listening paradigms to identify neural processing patterns that characterize dyslexic versus DLD children. Using a story listening paradigm, we show that atypical power dynamics and phase-amplitude coupling between delta and theta oscillations characterize dyslexic and DLD children groups, respectively. We further identify EEG common spatial patterns (CSP) during speech listening across delta, theta and beta oscillations describing dyslexic versus DLD children. A linear classifier using four deltaband CSP variables predicted dyslexia status (0.77 AUC). Crucially, these spatial patterns also identified children with dyslexia in a rhythmic syllable task EEG, suggesting a core developmental deficit in neural processing of speech rhythm. These findings suggest that there are distinct atypical neurocognitive mechanisms underlying dyslexia and DLD.

neuroscience↗

bioRxiv2022

Emergence of the cortical encoding of phonetic features in the first year of life

Even prior to producing their first words, infants are developing a sophisticated speech processing system, with robust word recognition present by 4-6 months of age. These emergent linguistic skills, observed with behavioural investigations, are likely to rely on increasingly sophisticated neural underpinnings. The infant brain is known to robustly track the speech envelope, however to date no cortical tracking study could investigate the emergence of phonetic feature encoding. Here we utilise temporal response functions computed from electrophysiological responses to nursery rhymes to investigate the cortical encoding of phonetic features in a longitudinal cohort of infants when aged 4, 7 and 11 months, as well as adults. The analyses reveal an increasingly detailed and acoustically-invariant phonetic encoding over the first year of life, providing the first direct evidence that the pre-verbal human cortex learns phonetic categories. By 11 months of age, however, infants still did not exhibit adult-like encoding.

neuroscience↗

bioRxiv2020

Neurophysiological indices of audiovisual speech integration are enhanced at the phonetic level for speech in noise

Seeing a speakers face benefits speech comprehension, especially in challenging listening conditions. This perceptual benefit is thought to stem from the neural integration of visual and auditory speech at multiple stages of processing, whereby movement of a speakers face provides temporal cues to auditory cortex, and articulatory information from the speakers mouth can aid recognizing specific linguistic units (e.g., phonemes, syllables). However it remains unclear how the integration of these cues varies as a function of listening conditions. Here we sought to provide insight on these questions by examining EEG responses to natural audiovisual, audio, and visual speech in quiet and in noise. Specifically, we represented our speech stimuli in terms of their spectrograms and their phonetic features, and then quantified the strength of the encoding of those features in the EEG using canonical correlation analysis. The encoding of both spectrotemporal and phonetic features was shown to be more robust in audiovisual speech responses then what would have been expected from the summation of the audio and visual speech responses, consistent with the literature on multisensory integration. Furthermore, the strength of this multisensory enhancement was more pronounced at the level of phonetic processing for speech in noise relative to speech in quiet, indicating that listeners rely more on articulatory details from visual speech in challenging listening conditions. These findings support the notion that the integration of audio and visual speech is a flexible, multistage process that adapts to optimize comprehension based on the current listening conditions. Significance StatementDuring conversation, visual cues impact our perception of speech. Integration of auditory and visual speech is thought to occur at multiple stages of speech processing and vary flexibly depending on the listening conditions. Here we examine audiovisual integration at two stages of speech processing using the speech spectrogram and a phonetic representation, and test how audiovisual integration adapts to degraded listening conditions. We find significant integration at both of these stages regardless of listening conditions, and when the speech is noisy, we find enhanced integration at the phonetic stage of processing. These findings provide support for the multistage integration framework and demonstrate its flexibility in terms of a greater reliance on visual articulatory information in challenging listening conditions.

neuroscience↗

bioRxiv2019

Cortical encoding of melodic expectations in human temporal cortex

Humans engagement in music rests on underlying elements such as the listeners cultural background and general interest in music, all shaping the way music is processed in the brain and perceived. Crucially, these factors modulate how listeners anticipate musical events, a process inducing instantaneous neural responses as the music confronts these expectations. Measuring such neural correlates would represent a direct window into high-level brain processing of music. Here we recorded electroencephalographic and electrocorticographic brain responses as participants listened to Bach melodies. We assessed the relative contributions of the acoustic versus melodic components of the music to the neural signal. Acoustic features included envelope and its derivative. Melodic features included information on melodic progressions (pitch) and their tempo (onsets), which were extracted from a Markov model predicting the next note based on a corpus of Western music and the preceding proximal musical context. We related the music to brain activity with a linear temporal response function, and demonstrated that cortical responses to music encode melodic expectations. Specifically, individual-subject neural signals were better predicted by a combination of acoustic and melodic expectation features than by either alone. This effect was most pronounced at response latencies up to 350ms, and in both planum temporale and Heschls gyrus. Finally, expectations of pitch and onset-time of musical notes exerted independent cortical effects, and such influences were modulated by the listeners musical expertise. Overall, this study demonstrates how the interplay of experimental and theoretical approaches can yield novel insights into the cortical encoding of melodic expectations.

neuroscience↗

bioRxiv2018

Accurate Modeling of Brain Responses to Speech

Perceptual processes can be probed by fitting stimulus-response models that relate measured brain signals such as electroencephalography (EEG) to the stimuli that evoke them. These models have also found application for the control of devices such as hearing aids. The quality of the fit, as measured by correlation, classification, or information rate metrics, indicates the value of the model and the usefulness of the device. Models based on Canonical Correlation Analysis (CCA) achieve a quality of fit that surpasses that of commonly-used linear forward and backward models. Here, we show that their performance can be further improved using several techniques that capture the time-varying and context-dependent relationships within the data, including adaptive beamforming, CCA weight optimization, and recurrent neural networks that capture the time-varying and context-dependent relationships within the data. We demonstrate these results using a match-vs-mismatch classification paradigm, in which the classifier must decide which of two stimulus samples produced a given EEG response and which is a randomly chosen stimulus sample. This task captures the essential features of the more complex auditory attention decoding (AAD) task explored in many other studies.

neuroscience↗

bioRxiv2018

Cortical responses to natural speech reflect probabilistic phonotactics

Humans comprehend speech despite the various challenges of real-world environments, such as loud noise and mispronunciation. Our auditory system is robust to these thanks to the integration of the upcoming sensory input with prior knowledge and expectations built on language-specific regularities. One such regularity regards the permissible phoneme sequences, which determine the likelihood that a word belongs to a given language (phonotactic probability; \"blick\" is more likely to be an English word than \"bnick\"). Previous research suggested that violations of these rules modulate brain evoked responses such as the N400 and the late positive complex. Yet several fundamental questions remain unresolved, especially regarding the neural encoding and integration strategy of phonotactic information. Here, we used linear modelling approaches to assess the influence of phonotactic probabilities on the brain responses to narrative speech measured with non-invasive EEG. We found that the relationship between continuous speech and EEG responses is best described when the speech descriptor includes phonotactic probabilities. This provides us with a methodology to isolate and measure the brain responses to phonotactics using natural speech at the individual subject-level. Furthermore, such low-frequency signals showed the strongest speech-EEG interactions at latencies of 100-400 ms, supporting a pre-lexical role of phonotactic information.\n\nSignificance StatementSpeech is composed of basic units, called phonemes, whose combinations comply with language-specific regularities determining whether a sequence \"sounds\" as a plausible word. Our ability to detect irregular combinations requires matching incoming sequences with our internal expectations, a process that supports speech segmentation and learning. However, the neural mechanisms underlying this phenomenon have not yet been established. Here, we examine this in the human brain using narrative speech. We identified a brain signal reflecting the likelihood that a word belongs to the language, which may offer new opportunities to investigate speech perception, learning, development, and impairment. Our data also suggest a pre-lexical role of this phenomenon, thus supporting and extending current mechanistic perspectives.

neuroscience↗

bioRxiv2017

Electrophysiological correlates of semantic dissimilarity reflect the comprehension of natural, narrative speech.

Understanding natural speech requires that the human brain convert complex spectrotemporal patterns of acoustic input into meaning in a rapid manner that is reasonably tightly time-locked to the incoming speech signal. However, neural evidence for such a time-locked process has been lacking. Here, we sought such evidence by using a computational model to quantify the meaning carried by each word based on how semantically dissimilar it was to its preceding context and then regressing this quantity against electroencephalographic (EEG) data recorded from subjects as they listened to narrative speech. This produced a prominent negativity at a time-lag of 200- 600 ms on centro-parietal EEG electrodes. Subsequent EEG experiments involving time-reversed speech, cocktail party attention and audiovisual speech-in-noise demonstrated that this response was exquisitely sensitive to whether or not subjects were understanding the speech they heard. These findings demonstrate that, when successfully comprehending natural speech, the human brain encodes meaning as a function of the amount of new information carried by each word in a relatively time-locked fashion.

neuroscience↗