Temporal Dynamics of EEG Decoding for Continuously Changing Visual Stimuli

Abstract

Multivariate analyses of M/EEG data are typically performed on neural responses time-locked to discrete stimulus onsets. Such designs usually reveal high decoding performance during the initial transient response (0-500 ms), which subsequently drops to a lower, sustained level. Here, we examined time-resolved EEG decoding of natural scene processing when scenes gradually enter the visual field without a clear onset. We created video sequences in which one scene category (e.g., a beach) smoothly transitioned into another category (e.g., a forest) by blending images from two categories into a single composite panorama and moving a square aperture across it. We then compared EEG decoding for the first scenes within the transitions, which appeared with a sudden onset, to the second scenes, which emerged gradually as the videos progressed. For the first scenes, we observed robust category decoding from 60 ms after onset with a clear peak structure. For the second scene, category decoding was markedly weaker and showed no discernable peak structure. Realigning the appearance of category-diagnostic content for the second scene using deep neural networks did not enhance decoding or recover a peak structure. Further, classifiers trained on the first scene generalized to the second, but with a broad, temporally diffuse pattern, indicating that the second scene did not engage the same hierarchical temporal cascade as the first. Together, these results demonstrate that sudden versus gradual onsets produce distinct temporal decoding dynamics. Insights from onset-based decoding studies, therefore, do not straightforwardly extend to continuous and free-flowing natural stimulation.