Representational Similarity and Pattern Classification of Fifteen Emotional States Induced by Movie Clips and Text Scenarios

Abstract

How do different emotional states relate to each other and how are they represented in the human brain? These are important questions in the field of affective neuroscience, with profound scientific and vast clinical implications. Different theories of emotion tend to emphasize the relative importance of distinct psychological constructs, for example categorical labels (e.g., fear, joy, or sadness) versus dimensional ratings (e.g., valence and arousal), for understanding human emotions. To investigate whether categorical or dimensional constructs correspond better to patterns of brain activity associated with human affective experiences, we experimentally induced 15 emotions (spanning positive, negative, and neutral valence) in 136 participants using 150 short movie clips and 150 one- or two-sentence text scenarios, while their blood oxygenation-level dependent activity was recorded in a magnetic resonance imaging scanner. Results from our representational similarity analyses suggest participants brain activity significantly correlated with their categorical labeling of, but not their dimensional rating of, the movie clips and text scenarios. Subsequently, we were also able to decode the categorical labels of these emotional stimuli using whole-brain multi-voxel pattern classification, with important voxels found in many cortical, limbic, subcortical, cerebellar, and brainstem regions. Finally, we found similar clusters of emotions through exploratory hierarchical clustering analyses of participants categorical labeling of and brain responses to these stimuli. Taken together, these findings greatly advance our understanding of how a large set of human emotions are related to each other both in terms of the participants self-report and their brain activity. Significance StatementWe successfully decoded fifteen emotional states induced by movie clips and text scenarios based on participants brain responses (blood oxygenation-level dependent signals) to these stimuli using multi-voxel pattern classification (a supervised machine learning approach). We also found that participants brain responses correlated with their self-reported emotional experience, i.e. which emotion they felt while they were presented with these stimuli, using representative similarity analysis (an unsupervised approach). Finally, we found that these emotions are organized in very similar ways both in terms of participants categorical labeling and their brain responses to these stimuli. Together, these data-driven and computational modeling-based findings greatly advance our understanding of how a large set of emotions are organized and represented in the human brain.