Dissociable effects of curiosity and hedonic valence on reinforcement learning

Abstract

Curiosity and exploration support learning and adaptive decision-making in uncertain environments. While these processes are sensitive to motivational context, it remains unclear how outcome valence shapes exploration across species. Human studies suggest that aversive contexts increase exploration, but these effects often rely on verbal framing and explicit instructions. To gain deeper insight into how exploration strategies emerge from experience alone, this study investigated the influence of hedonic valence on novelty seeking, exploration, and reinforcement learning in rhesus macaques. Using visual tokens as secondary reinforcers, we found that monkeys explored novel, uncertain options more frequently when exploitation would lead to losses rather than gains. However, our analyses clarified that this heightened novelty seeking was primarily a consequence of the monkeys employing an optimistic prior belief about the value of novelty, rather than a categorical, valence-dependent shift in their underlying curiosity or the information bonus associated with exploration. Approach and avoidance motivation did influence other aspects of reinforcement learning. Monkeys demonstrated faster learning from losses than from gains, indicating that they were averse to losing tokens. They also frequently chose an option and then quickly aborted their choice. These choice balks were strategic responses to approach-avoidance conflicts and uncertainty, and represented self-generated bouts of exploratory behavior that led to valence-dependent use of directed and random exploration. These findings suggest that different strategies are used to manage explore-exploit tradeoffs induced by novelty or internal motivational conflicts, revealing dissociable effects of curiosity and hedonic valence on reinforcement learning.