Search bioRxiv⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.64898/2026.03.01.708855

Multisensory coding of audiovisual movies in the human hippocampus

Raccah, O.·Agarwal, A.·Zhu, Y.·Turk-Browne, N. B.
View original source ↗Export metadata ↓

Abstract

The hippocampus receives convergent input from multiple sensory systems, yet in humans it has been studied almost exclusively through vision. Here we examine how the hippocampus contributes to sensory processing beyond the visual modality and to the multisensory integration of visual information with these other modalities. Participants were exposed to multiple repetitions of short naturalistic movie clips, each presented in four formats: auditory-only, visual-only, congruent audiovisual, and incongruent audiovisual (audio and video from different movies). Using high-resolution fMRI, we measured univariate activation and multivariate representations across the longitudinal axis and subfields of the human hippocampus. Whereas univariate analyses detected only visual responses across hippocampal subregions, with no activation for auditory stimuli and no benefit of congruent stimuli, multivariate analyses revealed robust representations of both auditory and visual scenes. The posterior hippocampus showed enhanced pattern similarity for congruent stimuli relative to unisensory stimuli, especially in the left hemisphere, demonstrating multisensory facilitation. The anterior hippocampus showed crossmodal decoding between auditory and visual versions of the same clip, driven by the right hemisphere, suggesting a more abstract representation. Finally, whole-brain searchlight analyses revealed parallel effects in cortical regions known to support multisensory integration. These findings advance understanding of auditory and multisensory coding in the human hippocampus, revealing specialization and lateralization along its longitudinal axis.

Explore related subjects

neurosciencePreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Raccah, O., Agarwal, A., Zhu, Y., Turk-Browne, N. B.. 2026-03-03. Multisensory coding of audiovisual movies in the human hippocampus. https://doi.org/10.64898/2026.03.01.708855

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Enhanced cortical tracking of unfamiliar languages in both monolinguals and bilinguals

Humans routinely encounter speech in languages they have never heard, yet how the brain responds to such input and whether bilingual experience shapes this response remains unknown. Here, we used electroencephalography (EEG) and temporal response function (TRF) modeling to examine cortical tracking of the speech envelope in 24 English-monolingual and 24 English-Mandarin bilingual adults. Participants listened to naturally produced continuous speech in three languages: English (familiar to all), Mandarin (familiar to bilinguals only), and Vietnamese (unfamiliar to all). We report two main findings. First, both monolinguals and bilinguals showed enhanced cortical tracking for unfamiliar relative to familiar languages, evidenced by higher EEG prediction accuracy (PA). Monolinguals showed enhanced tracking for both Mandarin and Vietnamese, whereas bilinguals showed enhancement only for Vietnamese, consistent with Mandarin being a familiar language for this group. This finding suggests that enhanced cortical encoding of unfamiliar speech is a general property of the listening brain, not a signature of listening to a non-native language or reduced language proficiency. Second, bilinguals strikingly showed stronger cortical tracking than monolinguals overall, in both PA and TRF peak weights, with the TRF peak weight advantage present across all three languages, suggesting a difference in how bilingual experience shapes the neural encoding of speech. These findings have implications for understanding how the brain navigates the linguistic diversity of everyday life in an increasingly global, multilingual world.

neuroscience↗
bioRxiv2026

Endosomal pH Triggers Amyloid β Oligomerization and Maladaptive Phenotypic Plasticity in Alzheimers Disease

Endosomal dysfunction is a presymptomatic hallmark of neurodegeneration. Recent evidence highlights dysregulation of endosomal pH as a central pathogenic hub in Alzheimer's disease (AD); however, the mechanisms linking pH shifts to neurodegeneration remain incompletely defined. Here, we use a quantitative model of endosomal acidification driven by proton pumping via the vacuolar ATPase, proton leak via the endosomal Na/H exchanger NHE6, and other ion-regulating elements. The model recapitulates how downregulation of NHE6 in AD promotes endosomal hyperacidification, potentially triggering maladaptive phenotypic plasticity, an initially adaptive response that becomes pathological. Analysis of human brain datasets reveals reciprocal enrichment of NHE6 in neurons and the related NHE9 in glia, with NHE6 co-expression networks enriched for synaptic signalling. Systematic curation of NHE6 patient variants indicates that loss-of-function is associated with late regression, consistent with progressive endosomal hyperacidification, supporting a conceptual framework where early compensation transitions to neurodegeneration. Mathematical analyses calibrated for neuronal endosomes reveal a saturable relationship between luminal pH and NHE6 dosage, with threshold-like behaviour below ~50% expression that hyperacidifies endosomes, correlating with AD severity. Our model suggests this pH shift may exponentially accelerate A{beta} oligomerization and enhance {beta}-secretase activity. Furthermore, A{beta} oligomerization estimates correlate with dysregulation of calcium signalling and synaptic dysfunction. Model findings are compared with experimental results from NHE6-null mice and a cell culture model of AD. Drawing parallels to cancer, we propose that endosomal pH serves as a conserved regulator of adaptive-to-maladaptive transitions. Restoring physiological endosomal pH may offer a therapeutic window to prevent irreversible neurodegeneration in AD.

neuroscience↗
bioRxiv2026

Diet Quality from Midlife to Later Life Relates to Late-Life Brain Health and Verbal Memory in the SG70 Cohort

Healthy diet across adulthood is associated with better late-life cognition, but how life-course diet quality relates to brain integrity, and whether brain measures mediate diet-cognition associations, remains unclear as studies with long-term diet records and detailed neurocognitive measures are lacking. We studied 892 participants from the SG70 study, nested within the Singapore Chinese Health Study, with adherence to the Dietary Approaches to Stop Hypertension diet (DASH) assessed between 1993--2025. Dietary quality during midlife, ages 44--55 years, and early elderhood, ages 61--73 years, was examined in relation to seven cognitive domains, brain morphometry, white matter hyperintensities and free-water MRI markers in late life, ages 68--82 years. Higher DASH adherence at both life stages was significantly associated with better late-ife verbal memory, and remained so when both life stages were modelled jointly. Higher midlife DASH adherence was associated with greater white matter volume in association tracts, whereas higher early-elderhood DASH adherence was associated with lower white matter hyperintensity (deep basal ganglia and anterior periventricular regions) and lower frontal and occipital grey matter free water, suggesting lower neurovascular and inflammatory burden. Mediation analyses indicated that white matter volume accounted for 12.3% in mediating the midlife DASH--verbal memory association, while cortical free water accounted for 12.5% in mediating the early-elderhood DASH--verbal memory association. Importantly, participants whose DASH adherence improved from lower adherence in midlife to better adherence in later life showed better verbal memory and more favourable brain integrity than those with persistently low adherence. These findings identify midlife and post-midlife diet quality as modifiable life-course exposures associated with late-life cognitive resilience through differences in macrostructural and microstructural brain integrity.

neuroscience↗