Embodied reinforcement learning in the primate cortico-basal ganglia system

Abstract

Learning the value of environmental stimuli from reward experience allows animals to make advantageous choices. Existing biological accounts of reinforcement learning (RL) often assume that this value is represented as a single, motor-system-invariant neural signal. Here we recorded neuronal activity across eight nodes of the macaque cortico-basal ganglia system -- from limbic to neocortical regions -- while monkeys learned stimulus-reward associations using either saccades or reaches as the required motor response. The geometry of population activity revealed largely distinct value-coding dimensions for saccades and reaches, including in limbic regions typically associated with motor-system-invariant value coding. Consequently, value information was substantially reduced when read out across motor systems. These results provide circuit-wide evidence that challenges existing neural implementations of RL. They link biological accounts of value learning to embodied frameworks in cognitive science and artificial intelligence, in which behavior is grounded in an agents physical structure rather than abstracted from it.