Learning Rates Are Not All the Same: The Interpretation of Computational Model Parameters Depends on the Context

Abstract

Reinforcement Learning (RL) models have revolutionized the cognitive and brain sciences, promising to explain behavior from simple conditioning to complex problem solving, to shed light on developmental and individual differences, and to anchor cognitive processes in specific brain mechanisms. However, the RL literature increasingly reveals contradictory results, which might cast doubt on these claims. We hypothesized that many contradictions arise from two commonly-held assumptions about computational model parameters that are actually often invalid: That parameters generalize between contexts (e.g., tasks, models) and that they capture interpretable (i.e., unique, distinctive) neurocognitive processes. To test this, we asked 291 participants aged 8-30 years to complete three learning tasks in one experimental session, and fitted RL models to each. We found that some parameters (exploration / decision noise) showed significant generalization: they followed similar developmental trajectories, and were reciprocally predictive between tasks. However, generalization was significantly below ceiling. Furthermore, other parameters (learning rates, forgetting) did not show evidence of generalization, and sometimes even opposite developmental trajectories. Interpretability was low for all parameters. We conclude that the systematic study of context factors (e.g., reward stochasticity; task volatility) will be necessary to enhance the generalizability and interpretability of computational cognitive models. HighlightsO_LIComputational cognitive modeling often makes two implicit assumptions: 1) Model parameters generalize between studies and models. 2) Parameters are interpretable, i.e., neurally, cognitively, and/or mechanistically specific. C_LIO_LIWe empirically investigate these assumptions in Reinforcement Learning (RL), using a large developmental dataset featuring three different learning tasks in a within-participant design. C_LIO_LIWe find that no parameters generalize between tasks in terms of absolute parameter values. However, RL decision noise/exploration parameters generalize in terms of between-participant variation, showing similar age trajectories across tasks. Interpretability was low for all parameters, especially those related to learning and memory. C_LIO_LIEven for parameters that showed generalization, the degree of generalization was significantly lower than in an artificial sample with perfect generalization for most parameters, revealing a relative lack of generalization. C_LIO_LIThese findings are in accordance with previous claims about the developmental trajectory of decision noise/exploration parameters, but suggest that claims about the development of learning rates or Forgetting might not be generalizable: these parameters are unlikely to capture the same neuro-cognitive processes across different tasks. C_LIO_LIWe suggest ways forward to improve the generalizability and interpretability of model parameters, in particular by focusing more on task context. C_LI