Valence-partitioned learning signals drive choice behavior and phenomenal subjective experience in humans

Abstract

How the human brain generates conscious phenomenal experience is a fundamental problem. In particular, it is unknown how variable and dynamic changes in subjective affect are driven by interactions with objective phenomena. We hypothesize a neurocomputational mechanism that generates valence-specific learning signals associated with what it is like to be rewarded or punished. Our hypothesized model maintains a partition between appetitive and aversive information while generating independent and parallel reward and punishment learning signals. This valence-partitioned reinforcement learning (VPRL) model and its associated learning signals are shown to predict dynamic changes in 1) human choice behavior, 2) phenomenal subjective experience, and 3) BOLD-imaging responses that implicate a network of regions that process appetitive and aversive information that converge on the ventral striatum and ventromedial prefrontal cortex during moments of introspection. Our results demonstrate the utility of valence-partitioned reinforcement learning as a neurocomputational basis for investigating mechanisms that may drive conscious experience. HighlightsO_LITD-Reinforcement Learning (RL) theory interprets punishments relative to rewards. C_LIO_LIEnvironmentally, appetitive and aversive events are statistically independent. C_LIO_LIValence-partitioned RL (VPRL) processes reward and punishment independently. C_LIO_LIWe show VPRL better accounts for human choice behavior and associated BOLD activity. C_LIO_LIVPRL signals predict dynamic changes in human subjective experience. C_LI