Search bioRxiv⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/2024.02.15.580589

Inverted encoding of neural responses to audiovisual stimuli reveals super-additive multisensory enhancement

Buhmann, Z.·Robinson, A. K.·Mattingley, J. B.·Rideaux, R.
View original source ↗Export metadata ↓

Abstract

A central challenge for the brain is how to combine separate sources of information from different sensory modalities to optimally represent objects and events in the external world, such as combining someones speech and lip movements to better understand them in a noisy environment. At the level of individual neurons, audiovisual stimuli often elicit super-additive interactions, where the neural response is greater than the sum of auditory and visual responses. However, investigations using electroencephalography (EEG) to record brain activity have revealed inconsistent interactions, with studies reporting a mix of super- and sub-additive effects. A possible explanation for this inconsistency is that standard univariate analyses obscure multisensory interactions present in EEG responses by overlooking multivariate changes in activity across the scalp. To address this shortcoming, we investigated EEG responses to audiovisual stimuli using inverted encoding, a population tuning approach that uses multivariate information to characterise feature-specific neural activity. Participants (n = 41) completed a spatial localisation task for both unisensory stimuli (auditory clicks, visual flashes) and combined audiovisual stimuli (spatiotemporally congruent clicks and flashes). To assess multivariate changes in EEG activity, we used inverted encoding to recover stimulus location information from event-related potentials (ERPs). Participants localised audiovisual stimuli more accurately than unisensory stimuli alone. For univariate ERP analyses we found an additive multisensory interaction. By contrast, multivariate analyses revealed a super-additive interaction [~]180 ms following stimulus onset, such that the location of audiovisual stimuli was decoded more accurately than that predicted by maximum likelihood estimation. Our results suggest that super-additive integration of audiovisual information is reflected within multivariate patterns of activity rather than univariate evoked responses.

Source connections

Explore related subjects

neurosciencePreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Buhmann, Z., Robinson, A. K., Mattingley, J. B., Rideaux, R.. 2024-02-17. Inverted encoding of neural responses to audiovisual stimuli reveals super-additive multisensory enhancement. https://doi.org/10.1101/2024.02.15.580589

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Neurodegeneration-inducing macromolecules exit the brain via nanovascular conduits formed by reticular fibroblasts

Accumulation of proteins such as amyloid beta (Abeta), hyperphosphorylated tau and alpha-synuclein within the brain alters neural information processing and causes neurodegeneration(1-3), but how toxic solutes are cleared from the brain remains highly controversial(4,5). Proposed exit routes include efflux across endothelial cells into the blood(6,7), and movement to the pial surface via vasomotion-induced pumping along spaces within arteriolar smooth muscle(8) or via outflow along the perivascular space of ascending venules promoted by water flux through astrocytes (the glymphatic system(9)). From the pial surface of the brain, drainage may continue to dural lymphatics, along the outer sheaths of exiting cranial nerves and across the cribriform plate(10-14). We now report the presence, in mice and humans, of 2 micron diameter conduits that remove fluorescently labelled tau and Abeta from the brain. These conduits form a spatially-organised mesh within the walls of penetrating arterioles and pial arteries, and around the surface of ascending venules and deep cerebral and pial veins. They course through the pial and arachnoid layers to span the CSF space, wrapping the brain and cranial nerves. They are formed of reticular fibroblasts, which label for VE-cadherin(15) and PDGFRalpha(16), the lymphatic markers(17) podoplanin, VEGFR3 and Prox1, and reticular fibroblast extracellular matrix components collagen I and VI(16,18-20). Parenchymal tau drains from the brain at a similar rate via arteriolar conduits and via conduits around venules, arguing against preferential removal by a glymphatic mechanism. In Alzheimer's disease model mice, Abeta is seen traversing these lymph node-like conduits. Modulation of molecular transfer via this route may accelerate or delay cognitive decline, and slowed transfer from arteriolar to pial-arachnoid conduits may initiate cerebral amyloid angiopathy.

neuroscience↗
bioRxiv2026

Analysis of the influence of gradual changes in matrix sentence similarity on neural envelope tracking

Neural tracking of speech is a well-established phenomenon in neuroscience. However, for speech signals with a fixed structure, significant correlations between speech envelopes and neurophysiological representations occur even for unheard sentences. We exploit a structured speech-in-noise matrix hearing test (Oldenburger Sentence Test, OLSA) to systematically quantify the relationship between acoustic sentence similarity and neural tracking. Simultaneous magnetoencephalography (MEG) and 76-channel electroencephalography (EEG) data, including 16 channels positioned directly around the ears (ear-EEG), were recorded from 21 young adults with normal hearing during the presentation of clean-speech audiobooks and OLSA sentences at six signal-to-noise ratios. A linear decoder trained on audiobooks reconstructed OLSA sentence envelopes. Reconstruction accuracies were compared using a linear mixed model across heard (matched) and unheard (mismatched) sentences of varying acoustic similarity. Significant reconstruction accuracies were achieved across MEG, EEG, and ear-EEG for both matched and mismatched sentences. For mismatched sentences, these accuracies gradually increased with their acoustic similarity to the heard speech data. The high similarity between sentences, which is especially prominent in matrix tests, can cause significant spurious tracking for mismatched stimuli. This effect can reach levels comparable to those of matched sentences and can be mistaken for true neural tracking. Robust neural tracking across modalities further supported the established viability of ear-EEG compared to whole-head systems.

neuroscience↗
bioRxiv2026

Seizures and tauopathy following neurotrauma are mediated by prion protein and metabotropic glutamate receptor 5

Traumatic brain injury (TBI) is one of the world's leading causes of death and disability and a major risk factor for dementias. The primary dementia associated with TBI is chronic traumatic encephalopathy (CTE), a neurodegenerative disease classified as a tauopathy, in which toxic tau molecules lead to disease pathologies and degeneration. The processes that lead to tauopathy and subsequent dementia after TBI remain unclear. Here, we built upon the finding that seizures after TBI may be a mechanism leading to tauopathy, by dissecting the functions of the metabotropic glutamate receptor 5 - cellular prion protein (mGluR5-PrPC) pathway. We delivered TBI to larval in a blast paradigm, and quantified aggregation of Tau via a genetically-encoded Tau-GFP fusion reporter. Zebrafish larvae lacking prp2 (homolog of mammalian cellular Prion Protein, PrPC) displayed a 168% increase in post-traumatic seizures activity after TBI. An mGluR5 agonist (CHPG) reduced post-traumatic seizures, whereas an mGluR5 antagonist (MPEP) increased post-traumatic seizures. Moreover, agonizing mGluR5 reduced tau aggregation and antagonizing mGluR5 increased tau burden. Larvae seizing from convulsants, rather than TBI, were treated with CHPG/MPEP and provided a similar pattern of outcomes, suggesting seizures may be a factor needed for mGluR5 activity to influence tau aggregation. The PrPC-mGluR5 pathway is proposed as one candidate pathomechanism linking TBI to subsequent seizures and tauopathy, and thus it warrants investigation as a target for prophylactic interventions.

neuroscience↗