Sustained dynamics and modality-specific network reconfigurations define crossmodal prediction error

Abstract

Predictive processing theories posit that the brain continuously generates expectations about incoming sensory inputs, updating them through prediction errors. While extensively studied within single modalities, it remains unclear how prediction errors unfold when expectations and violations occur across different senses. Using a local-global hierarchical oddball paradigm combined with high-density EEG in 47 participants, we contrasted unimodal and crossmodal prediction errors across auditory, visual, and somatosensory domains. We found that crossmodal violations elicit temporally sustained cortical responses which diverge from the transient, localised dynamics observed for unimodal prediction errors. Temporal decoding revealed that crossmodal effects maintain shared prolonged neural representations, suggestive of supramodal integration across all levels of cortical processing. Computational modelling further demonstrated that crossmodal prediction errors reorganize effective connectivity within and between sensory hierarchies, engaging distinct early cortical pathways depending on the sensory combination. Our findings refine hierarchical predictive coding for crossmodal transitions by demonstrating that, unlike unimodal prediction errors, crossmodal prediction errors recruit dedicated prolonged supramodal representations and flexibly adapted modality-specific networks.