Biology subjects

Eckstein, M. K.

Publications and source records attributed to Eckstein, M. K..

14 recordsLinked to original sources

bioRxiv2026

Reuse of learned subtask representations during human planning

The notion of a compositional reuse of previously learned problem solutions predicts that the human brain should reinstate such learned subtask representations during planning. Here, we test this in a value-ranking task implemented in an online and a magnetoencephalography (MEG) experiment, respectively. Participants first learned pairwise rankings and subsequently solved problems where these learned rankings could either be reused or required reordering. Behaviourally, in the online experiment, we show the availability of a reusable subtask structure improved accuracy, with computational modelling indicating a greater weighting of subtask-level value when participants could avail of this learnt structure. In MEG, during planning, we show enhanced reactivation of task-relevant subtask representations in the reusable condition, and particularly so early in the experiment. In a subsequent probe task, we show subtask-identity representations generalised to a novel context and that stimuli belonging to the same learned subtask manifest an increase in their neural similarity. Temporally delayed linear modelling (TDLM) revealed forward and backward sequential reactivation at a subtask level, alongside backward stimulus-level replay, with subtask sequenceness varying according to task context. The findings provide convergent behavioural, computational and neural evidence for reactivation and reuse of learnt relational structures for novel problem solving.

neuroscience↗

bioRxiv2026

AI-Discovered Cognitive Models Reveal Novel Insights into Human and Animal Learning

Scientific models are widely used across the natural sciences as an interface between scientific theories and empirical data [1]. Such models play a key role, for example, in the study of human and animal learning, where they express algorithmic hypotheses and relate them to psychology and neuroscience data [2, 3]. These models are traditionally handcrafted by expert researchers based on existing theory or new insights. Such handcrafted models, however, are now known to fall short of capturing the full richness of behavior, even in their narrow domains [4-7]. An alternative data-driven approach has emerged, seeking to discover new insights by fitting and interpreting flexible models [8-11]. However, these tools require substantial human effort to derive insight from data, and it has been unclear how to discover new ideas from data efficiently. Here, we present DataDIVER, a general approach for automatically discovering computational models from data, and demonstrate that these models surface novel mechanistic insights into human and animal learning. Our approach delivers models that take the form of short computer programs, which are optimized both to fit data well and to be simple. These programs explicitly connect with existing theoretical frameworks and are readily understandable by human scientists. They can also be used to make novel predictions, some of which we show are borne out in re-analysis of existing data. General-purpose tools for surfacing new ideas from data, especially in combination with the large datasets that are increasingly available in many fields, stand to dramatically accelerate scientific discovery.

animal behavior and cognition↗

bioRxiv2026

Functional reorganization of motor cortex connectivity during learning

Learning new tasks requires the brain to reshape the flow of neural activity, but how these changes arise from dynamic neural connectivity remains unclear. Here, we used two-photon photostimulation and calcium imaging to map learning-related changes in connectivity in layer 2/3 of mouse motor cortex, induced by learning of an optical brain-computer interface (BCI) task. Mice rapidly (within minutes) learned to change activity in a conditioned neuron to earn rewards. Activity changes were sparse; the conditioned neuron increased activity more than surrounding neurons. Mapping connectivity before and after learning revealed changes in motor cortex connectivity, enriched in neurons that were active before trial initiation, analogous to motor cortex populations that are active preceding movement. Motor cortex plasticity reroutes preparatory activity to neurons that are active later and control the conditioned neuron. Our findings show how rapid learning can be achieved through structured changes in motor cortex connectivity.

neuroscience↗

bioRxiv2025

Reasoning with programs in replay

Reasoning flexibly composes known elements to solve novel problems. Recent theories suggest the brain uses the axis of time to compose elements for reasoning. In this view, elements are packaged into fast neural sequences, with each sequence exploring the implications of a different composition. Using magnetoencephalography, we tested this idea while participants mentally executed programs. Each program contained a set of steps linked to operations, the implications of which had to be computed. We found that behavioral performance scaled with program complexity, and by the end of execution, inferred program solutions were represented in prefrontal and parietal cortices. We identified a possible mechanism by which these solutions were computed. During reasoning, representations of steps in the program reactivated in fast sequences, consistent with sampling candidate partial solutions. Further evidence suggested these reactivations were accompanied by representations of their operations, and were followed by neural patterns reflecting their computed implications. Together, these results suggest replaying sequences supports program execution and reveal a highly organized temporal microarchitecture of reasoning.

neuroscience↗

bioRxiv2025

Low dimensional latent structure underlying the choices of mice

An impressive wealth of cognitive neuroscience tasks involves combining perceptual information with an estimate of the latent state of the environment to make a decision. Such tasks have driven the development of theoretically-motivated cognitive models which offer compactly parameterised, and thereby insightful, accounts of the internal processes by which this might happen. However, the very large amounts of data that can now be collected present a challenge and an opportunity for this frame-work. The challenge is that these models can be shown to underfit the data systematically (Nassar & Frank, 2016; Palminteri, Wyart, & Koechlin, 2017; Wilson & Collins, 2019) - particularly in how they characterise the effect of the latent state estimate. But this opens up the opportunity to extend the framework and employ richer, more highly parameterised and flexible models, in a data-driven manner (Dezfouli, Griffiths, Ramos, Dayan, & Balleine, 2019). Unfortunately, it is tremendously difficult to interpret these models, precisely because of their flexibility. Here, we follow a recent approach (Eckstein, Summerfield, Daw, & Miller, 2024), in which components of compact models are progressively replaced with more flexible homologues, but, where possible, the resulting insights are mapped back into theoretically-transparent forms. We prove the effectiveness of our scheme by applying it to one of the largest available decision-making data sets for mice - more than 300,000 choices from 139 subjects studied by the International Brain Lab (The International Brain Laboratory et al., 2021). We found widely generalisable phenomena such as notable effects of continual fluctuations in task engagement and systematic differences between learning and forgetting of chosen and unchosen options in determining the subjective state estimate. Our results show that combining theory-driven and data-driven methods can reveal cognitive processes that would have been difficult to discover using either method on its own.

animal behavior and cognition↗

bioRxiv2025

Hybrid Neural-Cognitive Models Reveal Flexible Context-Dependent Information Processing in Reversal Learning

Reversal learning tasks provide a key paradigm for studying behavioral flexibility, requiring individuals to update choices in response to shifting reward contingencies. While reinforcement learning (RL) models have been widely used to provide interpretable explanations of human behavior on similar learning tasks, recent work has revealed that they often fail to fully account for the complexity of learning dynamics. In contrast, artificial neural networks (ANNs) often achieve substantially higher predictive accuracy, but lack the interpretability afforded by classical RL models. To bridge this gap, we introduce HybridRNNs--neural-cognitive models that integrate RL-inspired structures with flexible recurrent architectures. Among them, Context-ANN incorporates latent reward history and choice perseverance, and demonstrates improved alignment with human behavior compared to traditional RL models across two datasets. While not perfectly replicating human strategies, Context-ANN achieves comparable predictive accuracy to generic RNNs and offers interpretable value representations. Additional analyses of hidden dynamics reveal structured, context-sensitive internal states that adapt following reversals. These results suggest that humans may rely on more flexible or context-sensitive learning strategies even in simple reversal tasks, and highlight the potential of HybridRNNs as cognitive models that balance interpretability and predictive accuracy.

animal behavior and cognition↗

bioRxiv2025

Discovering Symbolic Cognitive Models from Human and Animal Behavior

Symbolic models play a key role in cognitive science, expressing computationally precise hypotheses about how the brain implements a cognitive process. Identifying an appropriate model typically requires a great deal of effort and ingenuity on the part of a human scientist. Here, we adapt FunSearch Romera-Paredes et al. (2024), a recently developed tool that uses Large Language Models (LLMs) in an evolutionary algorithm, to automatically discover symbolic cognitive models that accurately capture human and animal behavior. We consider datasets from three species performing a classic reward-learning task that has been the focus of substantial modeling effort, and find that the discovered programs outperform state-of-the-art cognitive models for each. The discovered programs can readily be interpreted as hypotheses about human and animal cognition, instantiating interpretable symbolic learning and decision-making algorithms. Broadly, these results demonstrate the viability of using LLM-powered program synthesis to propose novel scientific hypotheses regarding mechanisms of human and animal cognition.

animal behavior and cognition↗

bioRxiv2023

Nucleus accumbens dopamine release reflects Bayesian inference during instrumental learning

Dopamine release in the nucleus accumbens has been hypothesized to signal reward prediction error, the difference between observed and predicted reward, suggesting a biological implementation for reinforcement learning. Rigorous tests of this hypothesis require assumptions about how the brain maps sensory signals to reward predictions, yet this mapping is still poorly understood. In particular, the mapping is non-trivial when sensory signals provide ambiguous information about the hidden state of the environment. Previous work using classical conditioning tasks has suggested that reward predictions are generated conditional on probabilistic beliefs about the hidden state, such that dopamine implicitly reflects these beliefs. Here we test this hypothesis in the context of an instrumental task (a two-armed bandit), where the hidden state switches repeatedly. We measured choice behavior and recorded dLight signals reflecting dopamine release in the nucleus accumbens core. Model comparison among a wide set of cognitive models based on the behavioral data favored models that used Bayesian updating of probabilistic beliefs. These same models also quantitatively matched the dopamine measurements better than non-Bayesian alternatives. We conclude that probabilistic belief computation contributes to instrumental task performance in mice and is reflected in mesolimbic dopamine signaling.

animal behavior and cognition↗

bioRxiv2023

Predictive and Interpretable: Combining Artificial Neural Networks and Classic Cognitive Models to Understand Human Learning and Decision Making

Quantitative models of behavior are a fundamental tool in cognitive science. Typically, models are hand-crafted to implement specific cognitive mechanisms. Such "classic" models are interpretable by design, but may provide poor fit to experimental data. Artificial neural networks (ANNs), on the contrary, can fit arbitrary datasets at the cost of opaque mechanisms. Here, we adopt a hybrid approach, combining the predictive power of ANNs with the interpretability of classic models. We apply this approach to Reinforcement Learning (RL), beginning with classic RL models and replacing their components one-by-one with ANNs. We find that hybrid models can provide similar fit to fully-general ANNs, while retaining the interpretability of classic cognitive models: They reveal reward-based learning mechanisms in humans that are strikingly similar to classic RL. They also reveal mechanisms not contained in classic models, including separate rewardblind mechanisms, and the specific memory contents relevant to reward-based and reward-blind mechanisms.

neuroscience↗

bioRxiv2021

Learning Rates Are Not All the Same: The Interpretation of Computational Model Parameters Depends on the Context

Reinforcement Learning (RL) models have revolutionized the cognitive and brain sciences, promising to explain behavior from simple conditioning to complex problem solving, to shed light on developmental and individual differences, and to anchor cognitive processes in specific brain mechanisms. However, the RL literature increasingly reveals contradictory results, which might cast doubt on these claims. We hypothesized that many contradictions arise from two commonly-held assumptions about computational model parameters that are actually often invalid: That parameters generalize between contexts (e.g., tasks, models) and that they capture interpretable (i.e., unique, distinctive) neurocognitive processes. To test this, we asked 291 participants aged 8-30 years to complete three learning tasks in one experimental session, and fitted RL models to each. We found that some parameters (exploration / decision noise) showed significant generalization: they followed similar developmental trajectories, and were reciprocally predictive between tasks. However, generalization was significantly below ceiling. Furthermore, other parameters (learning rates, forgetting) did not show evidence of generalization, and sometimes even opposite developmental trajectories. Interpretability was low for all parameters. We conclude that the systematic study of context factors (e.g., reward stochasticity; task volatility) will be necessary to enhance the generalizability and interpretability of computational cognitive models. HighlightsO_LIComputational cognitive modeling often makes two implicit assumptions: 1) Model parameters generalize between studies and models. 2) Parameters are interpretable, i.e., neurally, cognitively, and/or mechanistically specific. C_LIO_LIWe empirically investigate these assumptions in Reinforcement Learning (RL), using a large developmental dataset featuring three different learning tasks in a within-participant design. C_LIO_LIWe find that no parameters generalize between tasks in terms of absolute parameter values. However, RL decision noise/exploration parameters generalize in terms of between-participant variation, showing similar age trajectories across tasks. Interpretability was low for all parameters, especially those related to learning and memory. C_LIO_LIEven for parameters that showed generalization, the degree of generalization was significantly lower than in an artificial sample with perfect generalization for most parameters, revealing a relative lack of generalization. C_LIO_LIThese findings are in accordance with previous claims about the developmental trajectory of decision noise/exploration parameters, but suggest that claims about the development of learning rates or Forgetting might not be generalizable: these parameters are unlikely to capture the same neuro-cognitive processes across different tasks. C_LIO_LIWe suggest ways forward to improve the generalizability and interpretability of model parameters, in particular by focusing more on task context. C_LI

neuroscience↗

bioRxiv2020

Modeling Changes in Probabilistic Reinforcement Learning during Adolescence

In the real world, many relationships between events are uncertain and probabilistic. Uncertainty is also likely to be a more common feature of daily experience for youth because they have less experience to draw from than adults. Some studies suggests probabilistic learning may be inefficient in youth compared to adults [1], while others suggest it may be more efficient in youth that are in mid adolescence [2, 3]. Here we used a probabilistic reinforcement learning task to test how youth age 8-17 (N = 187) and adults age 18-30 (N = 110) learn about stable probabilistic contingencies. Performance increased with age through early-twenties, then stabilized. Using hierarchical Bayesian methods to fit computational reinforcement learning models, we show that all participants performance was better explained by models in which negative outcomes had minimal to no impact on learning. The performance increase over age was driven by 1) an increase in learning rate (i.e. decrease in integration time horizon); 2) a decrease in noisy/exploratory choices. In mid-adolescence age 13-15, salivary testosterone and learning rate were positively related. We discuss our findings in the context of other studies and hypotheses about adolescent brain development. Author summaryAdolescence is a time of great uncertainty. It is also a critical time for brain development, learning, and decision making in social and educational domains. There are currently contradictory findings about learning in adolescence. We sought to better isolate how learning from stable probabilistic contingencies changes during adolescence with a task that previously showed interesting results in adolescents. We collected a relatively large sample size (297 participants) across a wide age range (8-30), to trace the adolescent developmental trajectory of learning under stable but uncertain conditions. We found that age in our sample was positively associated with higher learning rates and lower choice exploration. Within narrow age bins, we found that higher saliva testosterone levels were associated with higher learning rates in participants age 13-15 years. These findings can help us better isolate the trajectory of maturation of core learning and decision making processes during adolescence.

neuroscience↗

bioRxiv2020

Understanding the Unique Advantage of Adolescents in Stochastic, Volatile Environments: Combining Reinforcement Learning and Bayesian Inference

During adolescence, youth venture out, explore the wider world, and are challenged to learn how to navigate novel and uncertain environments. We investigated whether adolescents are uniquely adapted to this transition, compared to younger children and adults. In a stochastic, volatile reversal-learning task with a sample of 291 participants aged 8-30, we found that adolescents outperformed both younger and older participants. We developed two independent cognitive models, based on Reinforcement learning (RL) and Bayesian inference (BI). The RL parameter for learning from negative outcomes and the BI parameters specifying participants mental models peaked closest to optimal in adolescents, suggesting a central role in adolescent cognitive processing. By contrast, persistence and noise parameters improved monotonously with age. We distilled the insights of RL and BI using principal component analysis and found that three shared components interacted to form the adolescent performance peak: adult-like behavioral quality, child-like time scales, and developmentally-unique processing of positive feedback. This research highlights adolescence as a neurodevelopmental window that may be specifically adapted for volatile and uncertain environments. It also shows how detailed insights can be gleaned by using cognitive models in new ways.

neuroscience↗

bioRxiv2019

Computational evidence for hierarchically-structured reinforcement learning in humans

Humans have the fascinating ability to achieve goals in a complex and constantly changing world, still surpassing modern machine learning algorithms in terms of flexibility and learning speed. It is generally accepted that a crucial factor for this ability is the use of abstract, hierarchical representations, which employ structure in the environment to guide learning and decision making. Nevertheless, how we create and use these hierarchical representations is poorly understood. This study presents evidence that human behavior can be characterized as hierarchical reinforcement learning (RL). We designed an experiment to test specific predictions of hierarchical RL using a series of subtasks in the realm of context-based learning, and observed several behavioral markers of hierarchical RL, such as asymmetric switch costs between changes in higher-level versus lower-level features, faster learning in higher-valued compared to lower-valued contexts, and preference for higher-valued compared to lower-valued contexts. We replicated these results across three independent samples. We simulated three models: a classic RL, a hierarchical RL, and a hierar-chical Bayesian model, and compared their behavior to human results. While the flat RL model captured some aspects of participants sensitivity to outcome values, and the hierarchical Bayesian model some markers of transfer, only hierarchical RL accounted for all patterns observed in human behavior. This work shows that hierarchical RL, a biologically-inspired and computationally simple algorithm, can capture human behavior in complex, hierarchical environments, and opens the avenue for future research in this field.

neuroscience↗

bioRxiv2019

Distentangling the systems contributing to changes in learning during adolescence

Multiple neurocognitive systems contribute simultaneously to learning. For example, dopamine and basal ganglia (BG) systems are thought to support reinforcement learning (RL) by incrementally updating the value of choices, while the prefrontal cortex (PFC) contributes different computations, such as actively maintaining precise information in working memory (WM). It is commonly thought that WM and PFC show more protracted development than RL and BG systems, yet their contributions are rarely assessed in tandem. Here, we used a simple learning task to test how RL and WM contribute to changes in learning across adolescence. We tested 187 subjects ages 8 to 17 and 53 adults (25-30). Participants learned stimulus-action associations from feedback; the learning load was varied to be within or exceed WM capacity. Participants age 8-12 learned slower than participants age 13-17, and were more sensitive to load. We used computational modeling to estimate subjects use of WM and RL processes. Surprisingly, we found more robust changes in RL than WM during development. RL learning rate increased significantly with age across adolescence and WM parameters showed more subtle changes, many of them early in adolescence. These results underscore the importance of changes in RL processes for the developmental science of learning.\n\nHighlights- Subjects combine reinforcement learning (RL) and working memory (WM) to learn\n- Computational modeling shows RL learning rates grew with age during adolescence\n- When load was beyond WM capacity, weaker RL compensated less in younger adolescents\n- WM parameters showed subtler and more puberty-related changes\n- WM reliance, maintenance, and capacity had separable developmental trajectories\n- Underscores importance of RL processes in developmental changes in learning

neuroscience↗