Modulation of perception by visual, auditory, and audiovisual reward predicting cues

Abstract

Rewards influence information processing in the primary sensory areas specialized to process stimuli from a specific sensory modality. In real life situations, we receive sensory inputs not only from one single modality, but stimuli are often multisensory. It is however not known whether the reward-driven modulation of perception follows the same principles when reward is cued through a single or multiple sensory modalities. We previously showed that task-irrelevant reward cues modulate perception both intra- as well as cross-modally, likely through a putative enhancement in the integration of the stimulus parts into a coherent object. In this study, we explicitly test this possibility by assessing whether reward enhances the integration of unisensory components of a multisensory object in accordance with the supra-additive principle of multisensory integration. Towards this aim, we designed a simple detection task using reward predicting cues that were either unisensory (auditory or visual, both above the detection threshold) or multisensory (audiovisual). We conducted two experiments, behavioral (experiment 1) and simultaneous behavioral and neuroimaging testing (experiment 2). We expected that reward speeds up reaction times in response to all stimulus configurations, and that additionally the reward effects in multisensory cues fulfill the supra-additive principle of multisensory integration. We observed that reward decreased response times in both experiments with the strongest effect found for the multisensory stimuli in experiment 1. However, this behavioral effect did not fulfill the supra-additive principle. Neuroimaging results demonstrated sensory supra-additivity at the classical areas involved in multisensory integration such as the Superior Temporal areas (STS), while reward modulation was found in the midbrain and fronto-parietal areas, reflecting the typical areas that receive dopaminergic projections. However, reward did not enhance the supra-additivity in the STS compared to a no reward condition. Instead, we observed that some of the reward-related areas showed a sub-additive modulation by rewards and areas exhibiting a weaker supra-additive response to audiovisual stimuli, namely the fusiform gyrus, were modulated by rewards of audiovisual stimuli as measured by a conjunction analysis. Overall, our results indicate that reward does not enhance the multisensory integration through a supra-additive rule. These findings inspire a model where reward and sensory integration processes are regulated by two independent mechanisms, where sensory information is integrated at an early stage in a supra-additive manner, while reward modulates perception at a later stage sub-additively. Moreover, an associative area in the Fusiform gyrus exhibits a convergence of both reward and multisensory integration signals, indicating that it may be a hub to integrate different types of signals including rewards to disambiguate the information from different sensory modalities.