Search bioRxiv⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/2024.11.23.624953

Visual speech enhances auditory onset timing and envelope tracking through distinct mechanisms

Cao, C. Z.·Stacey, W. C.·Wasade, V. S.·Towle, V. L.·Tao, J. X.·Wu, S.·Issa, N. P.·Brang, D. J.
View original source ↗Export metadata ↓

Abstract

Seeing a speakers face facilitates speech recognition in challenging listening environments. Prior work has shown that visual speech contains timing information to aid auditory speech processing, yet how these signals are integrated within the auditory system during audiovisual speech perception remains poorly understood. Observation of preparatory mouth movements may initiate phase reset of intrinsic oscillations, potentially sensitizing the auditory system for receptive speech processing, while observation of mouth movements post speech onset may facilitate entrainment to the speech envelope. Yet, little work has been done to test whether visual speech enhances encoding of auditory speech onset, speech envelope tracking, or both, and through independent or overlapping mechanisms. To investigate this, we examined the ways in which visual speech timing information alters theta band power and phase using human intracranial electroencephalography (iEEG) recordings in a large group of patients with epilepsy (n = 21). Prior to speech onset, preparatory mouth movements elicited theta phase reset (increased inter-trial phase coherence; ITPC) throughout the superior temporal gyrus (STG), which is thought to enhance speech onset encoding. Following speech onset, visual speech modulated theta ITPC only at anterior STG electrodes while theta power was modulated at posterior STG electrodes. Pre- and post-speech onset were spatially and temporally dissociated, consistent with the hypothesis that audiovisual speech onset encoding and envelope tracking mechanisms are partially distinct. Crucially, congruent and incongruent visual speech, designed here to have identical visual timing information about speech onset time, but different visual mouth evolution, produced only a small difference in the phase of theta band oscillations in the anterior STG, highlighting a more restricted role of visual speech in ongoing auditory entrainment. These results support the hypothesis that visual speech improves the precision of auditory speech encoding through two separate mechanisms, with auditory speech onset encoded throughout the entire STG and ongoing speech envelope tracking within anterior STG.

Explore related subjects

neurosciencePreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Cao, C. Z., Stacey, W. C., Wasade, V. S., Towle, V. L., Tao, J. X., Wu, S., Issa, N. P., Brang, D. J.. 2024-11-24. Visual speech enhances auditory onset timing and envelope tracking through distinct mechanisms. https://doi.org/10.1101/2024.11.23.624953

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Exploring subacute effects of a DMT and harmine meditation retreat on the brain's control energy landscape

Modeling the brain as a controllable structural network allows the estimation of the control energy (CE) required to reach functional states, where low-CE states are more accessible. Psychedelics are thought to acutely flatten the brain's CE landscape, broadening the repertoire of reachable states, but whether this reorganization persists into the subacute ``afterglow" window is largely unknown. This exploratory study uses rs-fMRI from a randomized, double-blind, placebo-controlled trial (N = 39; 19 verum [N,N-dimethyltryptamine (DMT) + harmine, Ayahuasca-inspired formulation], 20 placebo) scanned before and after a three-day meditation retreat. We compute CE on a group human connectome project (HCP) structural connectome under two approaches that emphasize long and short timescale dynamics. Globally, we found no systematic shift in the distribution of CE between transitions, however mean CE increased in both groups at the long timescale approach. Surviving network effects emerge when a network's intrinsic timescale matches the stabilization regime. In the long timescale regime, the central executive network (CEN) showed an increase in post-retreat CE within the verum group that survived Benjamini-Hochberg False Discovery Rate (FDR-BH) correction. Although CE and functional connectivity are distinct, this higher CEN cost may relate to the reduced within-network connectivity previously reported subacutely after psilocybin. The default mode network (DMN) also showed a post-retreat CE increase in both groups, but neither change survived FDR correction. In the short timescale approach, the visual network (VN) showed a FDR-BH CE decrease within the verum group, consistent with reduced dynamical demand in a region acutely engaged by DMT. To examine whether changes were concentrated in less accessible states, we additionally analyzed rare brain states, defined by the top decile of information content. This analysis revealed a modest session x condition interaction in the long timescale approach. Overall, the two timescale lenses surface different network effects, that align with cortical timescale hierarchies. These findings point to network- and timescale-specific, rather than global, CE changes in the afterglow window, though effects are small and heterogeneous and require confirmation in larger samples. Characterizing whether and where psychedelic effects persist into the subacute window is relevant to psychedelic-assisted therapy, including safety, therapeutic timing, and durability of drug effects. More broadly, considering cortical timescale hierarchies may help identify network-specific CE changes that might be less apparent when networks are evaluated under a single dynamical timescale.

neuroscience↗
bioRxiv2026

Advanced meditation stabilizes cortical dynamics through inhibitory gain control linked to absorptive depth

Advanced meditation can shift conscious experience from ordinary body- and world-centered awareness toward absorption, detachment and dissociation-like states, yet the neural mechanisms that stabilize these immersive states remain unclear. We studied 18 long-term meditators during eyes-open rest and Rajyoga meditation, a practice in which practitioners reduce bodily identification while enhancing inner self-awareness. We combined multidimensional neural approach including frontal EEG spectral power, temporal non-stationarity and corticothalamic neural-field modeling, because these measures probe underlying complementary levels of the brain state transition: the strength of neural activity, its temporal stability, and the circuit level inhibition-excitation gain mechanisms. This framework is especially relevant to meditation because oscillatory, temporal organization may distinguish stable meditative states, and corticothalamic dynamics are closely linked to arousal and conscious state regulation. We correlated meditation induced neural changes with subjective ratings of absorption and dissociation. Compared with rest, meditation was associated with higher beta1, beta2 and gamma power, with no change in the slower bands (delta, theta and alpha), and with lower beta-band temporal non-stationarity, indicating a more stable expression of fast cortical dynamics. Fitting the corticothalamic model to the EEG spectra showed stronger cortical inhibitory gain and lower effective intracortical gain, placing meditation in an inhibition-dominant regime, with lower gain in the loop between excitatory and inhibitory neurons within the cortex. During meditation, these model estimates were related to subjective absorption and showed trend level links with dissociation. Together, these findings suggest that Rajyoga meditation stabilizes cortical dynamics through inhibitory gain control, linking absorption (deep and steady attention on the meditation) to dissociation-like self-detachment.

neuroscience↗
bioRxiv2026

CoReg, tracking-based motion compensation for in vivo calcium imaging

In vivo calcium imaging enables the observation of neural activity with high spatial and temporal resolution. However, recordings are frequently affected by motion artifacts caused by animal movement, physiological processes, tissue drift, or instability of the imaging setup. These artifacts make signal extraction harder and reduce the consistency of downstream analyses. Current registration methods can correct rigid and non-rigid motion, but often estimate motion from individual frames or image patches and make limited use of temporal information. In addition, activity-driven fluorescence changes can make it difficult to distinguish changes in image appearance from actual motion. Here, we treat motion correction in calcium imaging as a point tracking problem and introduce CoReg, an open-source Python- based software tool with a graphical user interface. We benchmark CoReg against established registration methods using fluorescence imaging datasets from multiple species. CoReg performs competitively with or better than existing approaches across several settings. We further show that its performance is robust to the choice of tracked points, making the method straightforward to apply. CoReg includes implementations of both our method and the evaluated established methods. Our results show that tracking-based registration provides a simple and reliable approach for motion correction in vivo calcium imaging.

neuroscience↗