Biology subjects

Rothenhoefer, K. M.

Publications and source records attributed to Rothenhoefer, K. M..

4 recordsLinked to original sources

bioRxiv2025

Dissociable effects of curiosity and hedonic valence on reinforcement learning

Curiosity and exploration support learning and adaptive decision-making in uncertain environments. While these processes are sensitive to motivational context, it remains unclear how outcome valence shapes exploration across species. Human studies suggest that aversive contexts increase exploration, but these effects often rely on verbal framing and explicit instructions. To gain deeper insight into how exploration strategies emerge from experience alone, this study investigated the influence of hedonic valence on novelty seeking, exploration, and reinforcement learning in rhesus macaques. Using visual tokens as secondary reinforcers, we found that monkeys explored novel, uncertain options more frequently when exploitation would lead to losses rather than gains. However, our analyses clarified that this heightened novelty seeking was primarily a consequence of the monkeys employing an optimistic prior belief about the value of novelty, rather than a categorical, valence-dependent shift in their underlying curiosity or the information bonus associated with exploration. Approach and avoidance motivation did influence other aspects of reinforcement learning. Monkeys demonstrated faster learning from losses than from gains, indicating that they were averse to losing tokens. They also frequently chose an option and then quickly aborted their choice. These choice balks were strategic responses to approach-avoidance conflicts and uncertainty, and represented self-generated bouts of exploratory behavior that led to valence-dependent use of directed and random exploration. These findings suggest that different strategies are used to manage explore-exploit tradeoffs induced by novelty or internal motivational conflicts, revealing dissociable effects of curiosity and hedonic valence on reinforcement learning.

animal behavior and cognition↗

bioRxiv2024

Contribution of amygdala to dynamic model arbitration under uncertainty

Intrinsic uncertainty in the reward environment requires the brain to run multiple models simultaneously to predict outcomes based on preceding cues or actions, commonly referred to as stimulus- and action-based learning. Ultimately, the brain also must adopt appropriate choice behavior using reliability of these models. Here, we combined multiple experimental and computational approaches to quantify concurrent learning in monkeys performing tasks with different levels of uncertainty about the model of the environment. By comparing behavior in control monkeys and monkeys with bilateral lesions to the amygdala or ventral striatum, we found evidence for dynamic, competitive interaction between stimulus-based and action-based learning, and for a distinct role of the amygdala. Specifically, we demonstrate that the amygdala adjusts the initial balance between the two learning systems, thereby altering the interaction between arbitration and learning that shapes the time course of both learning and choice behaviors. This novel role of the amygdala can account for existing contradictory observations and provides testable predictions for future studies into circuit-level mechanisms of flexible learning and choice under uncertainty.

neuroscience↗

bioRxiv2020

Transcriptional Diversity of Medium Spiny Neurons in the Primate Striatum

The striatum is the interface between dopamine reward signals and cortico-basal ganglia circuits that mediate diverse behavioral functions. Medium spiny neurons (MSNs) constitute the vast majority of striatal neurons and are traditionally classified as direct- or indirect-pathway neurons. However, that traditional model does not explain the anatomical and functional diversity of MSNs. Here, we defined molecularly distinct MSN types in the primate striatum, including (1) dorsal striatum MSN types associated with striosome and matrix compartments, (2) ventral striatum types associated with the nucleus accumbens shell and olfactory tubercle, and (3) an MSN-like type restricted to -opioid receptor rich islands in the ventral striatum. These results lay the foundation for achieving cell type-specific transgenesis in the primate striatum and provide a blueprint for investigating circuit-specific processing.

neuroscience↗

bioRxiv2019

Rare Rewards Amplify Dopamine Learning Responses

Dopamine neurons drive learning by coding reward prediction errors (RPEs), which are formalized as subtractions of predicted values from reward values. Subtractions accommodate point estimate predictions of value, such as the average value. However, point estimate predictions fail to capture many features of choice and learning behaviors. For instance, reaction times and learning rates consistently reflect higher moments of probability distributions. Here, we demonstrate that dopamine RPE responses code probability distributions. We presented monkeys with rewards that were drawn from the tails of normal and uniform reward size distributions to generate rare and common RPEs, respectively. Behavioral choices and pupil diameter measurements indicated that monkeys learned faster and registered greater arousal from rare RPEs, compared to common RPEs of identical magnitudes. Dopamine neuron recordings indicated that rare rewards amplified RPE responses. These results demonstrate that dopamine responses reflect probability distributions and suggest a neural mechanism for the amplified learning and enhanced arousal associated with rare events.

neuroscience↗