Similarity in sensory modality and information domain impair processing in a dual-task context: Evidence from behavior, pupillometry, and EEG

Abstract

Project AbstractMaking sense of our environment requires us to extract simultaneous temporal and spatial information from multiple sensory modalities, particularly audition and vision. This sensory information can be stored in working memory (WM) to guide future actions, at which point it must be safeguarded against interference from ongoing sensory processing. Recent fMRI research has uncovered regions in human frontal cortex well-suited to coordinate this interplay between attention and WM for multisensory and multidimensional information. Which of these brain regions are engaged depends on both the sensory modality of the input and the information domain of the task, forming the basis of two complementary networks specialized for auditory/temporal and visual/spatial processing. Motivated by the functional specializations of these networks, we examined whether similarity in sensory modality and information domain modulates neural and perceptual interference between two concurrent tasks. Participants stored temporal or spatial information about auditory or visual stimuli in WM, and on some trials, performed an intervening temporal or spatial auditory task during WM retention. WM recall and auditory perceptual judgments were impaired when the two tasks relied on the same sensory modality and/or information domain. Pupil dilations were also larger in these conditions, indicating increased cognitive effort. Event-related potentials (ERPs) revealed a neural signature of domain-based interference that was masked by behavioral ceiling effects. These results demonstrate that modality and information domain jointly affect how task information is represented in WM, and concomitantly, how tasks engage the complementary auditory-temporal and visual/spatial cognitive control networks.