Biology subjects

Daw, N. D.

Publications and source records attributed to Daw, N. D..

At least 19 recordsLinked to original sources

bioRxiv2026

AI-Discovered Cognitive Models Reveal Novel Insights into Human and Animal Learning

Scientific models are widely used across the natural sciences as an interface between scientific theories and empirical data [1]. Such models play a key role, for example, in the study of human and animal learning, where they express algorithmic hypotheses and relate them to psychology and neuroscience data [2, 3]. These models are traditionally handcrafted by expert researchers based on existing theory or new insights. Such handcrafted models, however, are now known to fall short of capturing the full richness of behavior, even in their narrow domains [4-7]. An alternative data-driven approach has emerged, seeking to discover new insights by fitting and interpreting flexible models [8-11]. However, these tools require substantial human effort to derive insight from data, and it has been unclear how to discover new ideas from data efficiently. Here, we present DataDIVER, a general approach for automatically discovering computational models from data, and demonstrate that these models surface novel mechanistic insights into human and animal learning. Our approach delivers models that take the form of short computer programs, which are optimized both to fit data well and to be simple. These programs explicitly connect with existing theoretical frameworks and are readily understandable by human scientists. They can also be used to make novel predictions, some of which we show are borne out in re-analysis of existing data. General-purpose tools for surfacing new ideas from data, especially in combination with the large datasets that are increasingly available in many fields, stand to dramatically accelerate scientific discovery.

animal behavior and cognition↗

bioRxiv2026

Meta-learning is expressed through altered prefrontal cortical dynamics

Learning where and when rewards like food and water are available is essential for survival1,2. In the simplest cases where resource availability is stable, animals can learn reward contingencies by integrating outcomes across repeated samples of each possible action. In more natural settings, however, reward availability is governed by structured higher-order rules such as depletion and repletion over time. To adapt flexibly to such changing environments, optimal choices require meta-learning wherein animals learn how to learn from external feedback, ultimately enabling them to infer the underlying reward structure from abstract, generalizable rules rather than relying solely on recent outcomes3,4. The existence of meta-learning in animal behavior is well established3-8, yet the neural circuits and computations that implement it remain poorly understood9-11. Here we investigated meta-learning using a spatial foraging task in which rats acquired a depletion-repletion rule that regulated reward availability, and carried out longitudinal, high-density recordings from the medial prefrontal cortex (mPFC). We show that meta-learning engages specific, systematic changes in mPFC neural dynamics that embed the learned rule and thereby alter how the network learns action values from reward outcomes. These dynamics are based on mixed coding of task structure and value in individual mPFC neurons. At the population level, this coding organizes into low-dimensional dynamical motifs that generalize across task conditions. As meta-learning progresses, these motifs are reshaped to instantiate both rule-guided inference of future states before outcome delivery and rule-based value updating during the outcome period. These results indicate that meta-learning sculpts pre-existing prefrontal dynamics to support the acquisition of new, generalizable reward-learning strategies.

neuroscience↗

bioRxiv2025

Dissecting the contribution of recent reward versus recent performance history on cognitive effort allocation

An extensive body of literature has shown that humans tend to avoid expending cognitive effort, just like for physical effort or financial resources. How then, do we decide whether to put this effort in? Decision-making not only involves choosing our actions, but also the meta-decision of how much cognitive effort to invest in making this choice, weighing the costs of cognitive effort against potential rewards. Popular recent theories, grounded in the field of reinforcement learning, suggest that this cost-benefit trade-off can be informed by the opportunity costs of effort investment, which the brain may approximate by the estimated average reward rate per unit time. It follows from intuition that in a low reward environment, investing cognitive resources in the task at hand will less likely lead to missed opportunities. Recent studies provided support for this idea, showing that people exert more cognitive effort when reward rate is low. Here, we replicate one of the key previous findings but provide an important nuance to this result. Cognitive effort allocation was better explained by participants recent performance history (i.e. accuracy rate) than average reward rate. In combination with the observation that participants were insensitive to the reward currently at stake, this invites a reinterpretation of these previous findings and suggests the need for further studies to assess whether environmental richness may indeed serve as a heuristic to modulate cognitive effort allocation.

neuroscience↗

bioRxiv2025

Error-driven changes in hippocampal representations accompany flexible re-learning

Flexible behavior requires both the learning of new associations, and the suppression of previous ones, but how neural circuits achieve this balance remains unclear. Here we show that continuous changes in hippocampal representations, known as drift, may facilitate this process. We used voluntary head-fixation and calcium imaging to record from CA1 in rats during an odor-guided navigation task that required frequent re-learning. We found systematic representational changes over the course of the multi-hour sessions that were increased following errors. A simple neural network model revealed that such error-driven drift can enable flexible re-learning by allowing new associations to form from new neural patterns. A consequence of this is that previous associations are maintained in latent synaptic weights. These findings reconcile the apparent tension between representational drift and stable memory storage, demonstrating how dynamic neural codes could support both flexible behavior and lasting memories.

neuroscience↗

bioRxiv2025

Discovering Symbolic Cognitive Models from Human and Animal Behavior

Symbolic models play a key role in cognitive science, expressing computationally precise hypotheses about how the brain implements a cognitive process. Identifying an appropriate model typically requires a great deal of effort and ingenuity on the part of a human scientist. Here, we adapt FunSearch Romera-Paredes et al. (2024), a recently developed tool that uses Large Language Models (LLMs) in an evolutionary algorithm, to automatically discover symbolic cognitive models that accurately capture human and animal behavior. We consider datasets from three species performing a classic reward-learning task that has been the focus of substantial modeling effort, and find that the discovered programs outperform state-of-the-art cognitive models for each. The discovered programs can readily be interpreted as hypotheses about human and animal cognition, instantiating interpretable symbolic learning and decision-making algorithms. Broadly, these results demonstrate the viability of using LLM-powered program synthesis to propose novel scientific hypotheses regarding mechanisms of human and animal cognition.

animal behavior and cognition↗

bioRxiv2025

A Rational Information Gathering Account of Infant Habituation

Gaze is one of the primary experimental measures for studying cognitive development, especially in preverbal infants. However, the field is only beginning to develop a principled explanatory framework for making sense of the various factors affecting gaze. We approach this issue by addressing infant gaze from first principles, using rational information gathering. In particular, we revisit the influential descriptive account of Hunter and Ames (1988) (H&A), which posits a set of regularities argued to govern how gaze preference for a stimulus changes with experience and other factors. When the H&As model is reconsidered from the perspective of rational information gathering (as recently also proposed by other authors), one feature of the model emerges as surprising: that preference for a stimulus is not monotonic with exposure. This claim, which has at least some empirical support, is in contrast to most statistical measures of informativeness, which strictly decline with experience. We present a normative, computational theory of visual exploration that rationalizes this and other features of the classic account. Our account suggests that H&As signature nonmonotonic pattern is a direct manifestation of a ubiquitous principle of the value of information in sequential tasks, other consequences of which have recently been observed in a range of settings including deliberation, exploration, curiosity, and boredom. This is that the value of information gathering, putatively driving gaze, depends on the interplay of a stimulus informativeness (called Gain, the focus of other rationally motivated accounts) with a second factor (called Need) reflecting the estimated chance that information will be used in the future. This computational decomposition draws new connections between infant gaze and other cases of exploration, and offers novel, quantitative interpretations and predictions about the factors that may impact infant exploratory attention.

animal behavior and cognition↗

bioRxiv2024

Trial-by-trial learning of successor representations in human behavior

Decisions in humans and other organisms depend, in part, on learning and using models that capture the statistical structure of the world, including the long-run expected outcomes of our actions. One prominent approach to forecasting such long-run outcomes is the successor representation (SR), which predicts future states aggregated over multiple timesteps. Although much behavioral and neural evidence suggests that people and animals use such a representation, it remains unknown how they acquire it. It has frequently been assumed to be learned by temporal difference bootstrapping (SR-TD(0)), but this assumption has largely not been empirically tested or compared to alternatives including eligibility traces (SR-TD({lambda} > 0)). Here we address this gap by leveraging trial-by-trial reaction times in graph sequence learning tasks, which are favorable for studying learning dynamics because the long horizons in these studies differentiate the transient update dynamics of different learning rules. We examined the behavior of SR-TD({lambda}) on a probabilistic graph learning task alongside a number of alternatives, and found that behavior was best explained by a hybrid model which learned via SR-TD({lambda}) alongside an additional predictive model of recency. The relatively large{lambda} we estimate indicates a predominant role of eligibility trace mechanisms over the bootstrap-based chaining typically assumed. Our results provide insight into how humans learn predictive representations, and demonstrate that people simultaneously learn the SR alongside lower-order predictions. Author SummaryOur ability to plan intelligently requires predicting the state of the world multiple steps into the future. Enumerating future outcomes step-by-step, however, is slow and costly. Instead, research has shown that people rely on simplified models of the world that skip across multiple steps at once. How do we construct these simplified models? One promising idea is the successor representation (SR), which predicts future events via a simple and neurally plausible computation. The SR has been shown to explain a range of behavioral phenomena, but these studies have not identified which among many learning rules the brain uses to build the SR. Plausible mechanisms for learning associations over delays (called bootstrapping and eligibility traces) both converge to identical simplified world models, and thus existing studies on the SR, which focus on well trained behavior, are unable to distinguish between them. Here, we answer this question by examining behavior on a graph learning task, where stimulus-by-stimulus reaction times have been shown to reflect predictions over long temporal horizons. Through both model fitting and model-agnostic comparisons, we find that behavior is best explained by a learning rule heavily dependent on eligibility traces, in contrast to previous work which generally assumed an (untested) bootstrapping update rule.

neuroscience↗

bioRxiv2024

Active reconfiguration of neural task states

The ability to switch between tasks is a core component of human intelligence, yet a mechanistic understanding of this capacity has remained elusive. Long-standing debates over how task switching is influenced by preparation for upcoming tasks or interference from previous tasks have been difficult to resolve without quantitative neural predictions. We advance this debate by using state-space modeling to directly compare the latent task dynamics in task-optimized recurrent neural networks and human electroencephalographic recordings. Over the inter-trial interval, both networks and brains converged into a neutral task state, a novel control strategy that reconciles the role of preparation and interference in task switching. These findings provide a quantitative account of cognitive flexibility and a promising paradigm for bridging artificial and biological neural networks.

neuroscience↗

bioRxiv2024

Hippocampal representations of alternative possibilities are flexibly generated to meet cognitive demands

The cognitive ability to go beyond the present to consider alternative possibilities, including potential futures and counterfactual pasts, can support adaptive decision making. Complex and changing real-world environments, however, have many possible alternatives. Whether and how the brain can select among them to represent alternatives that meet current cognitive needs remains unknown. We therefore examined neural representations of alternative spatial locations in the rat hippocampus during navigation in a complex patch foraging environment with changing reward probabilities. We found representations of multiple alternatives along paths ahead and behind the animal, including in distant alternative patches. Critically, these representations were modulated in distinct patterns across successive trials: alternative paths were represented proportionate to their evolving relative value and predicted subsequent decisions, whereas distant alternatives were prevalent during value updating. These results demonstrate that the brain modulates the generation of alternative possibilities in patterns that meet changing cognitive needs for adaptive behavior.

neuroscience↗

bioRxiv2024

Reconciling Flexibility and Efficiency: Medial Entorhinal Cortex Represents a Compositional Cognitive Map

The influential concept of a cognitive map envisions that the brain builds mental representations of objects, barriers, and goals. This idea has been formalized in a range of computational models that show how such representations can be useful for guiding goal-directed behavior, for instance by planning novel routes to maximize long-run rewards. One key feature of flexible cognitive representations generally is that they exploit compositionality - the ability to build complex structures by recombining simpler parts. However, how this principle plays out in neural representations of cognitive maps and map-based planning remains largely unexplored. Indeed, as we show here, compositionality can be difficult to reconcile with efficient planning: because reuse tends to oppose flexibility, it is challenging to construct a compositional representation of the environment which is also organized in a way that enables generalizability and efficient planning. Here, we propose a novel model for efficiently creating and planning with compositional predictive maps, and further show that it successfully simulates various aspects of response fields in the medial entorhinal cortex, particularly object vector cells and grid cells. The model treats each object as an alteration to a baseline map linked to open space, creating complete predictive maps by combining object-related representations compositionally. Overall, this work provides a comprehensive and realistic model for efficient model learning and model-based planning in animals, and offers insights into the brain processes supporting efficient, flexible planning using compositional predictive maps.

neuroscience↗

bioRxiv2024

Prioritizing replay when future goals are unknown

Although hippocampal place cells replay nonlocal trajectories, the computational function of these events remains controversial. One hypothesis, formalized in a prominent reinforcement learning account, holds that replay plans routes to current goals. However, recent puzzling data appear to contradict this perspective by showing that replayed destinations lag current goals. These results may support an alternative hypothesis that replay updates route information to build a "cognitive map." Yet no similar theory exists to formalize this view, it is unclear how such a map is represented or what role replay plays in computing it. We address these gaps by introducing a theory of replay that learns a map of routes to candidate goals, before reward is available or when its location may change. Replay is then focused on current goals (as with planning) and/or potential future goals (like a map), depending on the animals expectations about future goal switching. Our work thus generalizes the planning account to capture a general map-building function for replay, reconciling it with data, and revealing an unexpected relationship between the seemingly distinct hypotheses. The theory offers a unifying explanation why data from tasks with different goal dynamics have seemingly supported different hypotheses for the function of replay, and suggests new predictions for experiments testing these effects. Graphical Abstract HighlightsO_LIA reinforcement learning model explains how to use replay to build a cognitive map for navigation C_LIO_LIMap-building replay explains recent replay data from goal-switching tasks that stymie previous theoretical accounts C_LIO_LIModel predicts replay in the absence of rewards as well as sensitivity of replay to goal statistics C_LIO_LIModel suggests planning-like replay is a special case of map-like replay C_LI

neuroscience↗

bioRxiv2024

Pre-existing visual responses in a projection-defined dopamine population explain individual learning trajectories

Learning a new task is challenging because the world is high dimensional, with only a subset of features being reward-relevant. What neural mechanisms contribute to initial task acquisition, and why do some individuals learn a new task much more quickly than others? To address these questions, we recorded longitudinally from dopamine (DA) axon terminals in mice learning a visual task. Across striatum, DA responses tracked idiosyncratic and side-specific learning trajectories. However, even before any rewards were delivered, contralateral-side-specific visual responses were present in DA terminals only in the dorsomedial striatum (DMS). These pre-existing responses predicted the extent of learning for contralateral stimuli. Moreover, activation of these terminals improved contralateral performance. Thus, the initial conditions of a projection-specific and feature-specific DA signal help explain individual learning trajectories. More broadly, this work implies that functional heterogeneity across DA projections serves to bias target regions towards learning about different subsets of task features, providing a mechanism to address the dimensionality of the initial task learning problem.

neuroscience↗

bioRxiv2024

Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning.

Different brain systems have been hypothesized to subserve multiple "experts" that compete to generate behavior. In reinforcement learning, two general processes, one model-free (MF) and one model-based (MB), are often modeled as a mixture of agents (MoA) and hypothesized to capture differences between automaticity vs. deliberation. However, shifts in strategy cannot be captured by a static MoA. To investigate such dynamics, we present the mixture-of-agents hidden Markov model (MoA-HMM), which simultaneously learns inferred action values from a set of agents and the temporal dynamics of underlying "hidden" states that capture shifts in agent contributions over time. Applying this model to a multi-step, reward-guided task in rats reveals a progression of within-session strategies: a shift from initial MB exploration to MB exploitation, and finally to reduced engagement. The inferred states predict changes in both response time and OFC neural encoding during the task, suggesting that these states are capturing real shifts in dynamics.

neuroscience↗

bioRxiv2024

Building compositional tasks with shared neural subspaces

Cognition is remarkably flexible; we are able to rapidly learn and perform many different tasks1. Theoretical modeling has shown artificial neural networks trained to perform multiple tasks will re-use representations2 and computational components3 across tasks. By composing tasks from these sub-components, an agent can flexibly switch between tasks and rapidly learn new tasks4. Yet, whether such compositionality is found in the brain is unknown. Here, we show the same subspaces of neural activity represent task-relevant information across multiple tasks, with each task compositionally combining these subspaces in a task-specific manner. We trained monkeys to switch between three compositionally related tasks. Neural recordings found task-relevant information about stimulus features and motor actions were represented in subspaces of neural activity that were shared across tasks. When monkeys performed a task, neural representations in the relevant shared sensory subspace were transformed to the relevant shared motor subspace. Subspaces were flexibly engaged as monkeys discovered the task in effect; their internal belief about the current task predicted the strength of representations in task-relevant subspaces. In sum, our findings suggest that the brain can flexibly perform multiple tasks by compositionally combining task-relevant neural representations across tasks.

neuroscience↗

bioRxiv2023

Proactive and reactive construction of memory-based preferences

We are often faced with decisions we have never encountered before, requiring us to infer possible outcomes before making a choice. Computational theories suggest that one way to make these types of decisions is by accessing and linking related experiences stored in memory. Past work has shown that such memory-based preference construction can occur at a number of different timepoints relative to the moment a decision is made. Some studies have found that memories are integrated at the time a decision is faced (reactively) while others found that memory integration happens earlier, when memories were initially encoded (proactively). Here we offer a resolution to this inconsistency, demonstrating that these two strategies tradeoff rationally as a function of the associative structure of memory. We use fMRI to decode patterns of brain responses unique to categories of images in memory and find that proactive memory access is more common and allows more efficient inference. However, we also find that participants use reactive access when choice options are linked to a larger number of memory associations. Together, these results indicate that the brain judiciously conducts proactive inference by accessing memories ahead of time when conditions make this strategy more favorable.

neuroscience↗

bioRxiv2023

Humans rationally balance mental simulation and temporally-abstract heuristics

How do people model the worlds dynamics to guide mental simulation and evaluate choices? One prominent approach, the Successor Representation (SR), takes advantage of temporal abstraction of future states: by aggregating trajectory predictions over multiple timesteps, the brain can avoid the costs of iterative, multi-step mental simulation. Human behavior broadly shows signatures of such temporal abstraction, but finer-grained characterization of individuals strategies and their dynamic adjustment remains an open question. We developed a novel task to measure SR usage during dynamic, trial-by-trial learning. Using this approach, we find that participants exhibit a mix of SR and model-based learning strategies that varies across individuals. Further, by dynamically manipulating the task contingencies within-subject to favor or disfavor temporal abstraction, we observe evidence of resource-rational reliance on the SR, which decreases when future states are less predictable. Our work adds to a growing body of research showing that the brain arbitrates between approximate decision strategies. The current study extends these ideas from simple habits into usage of more sophisticated approximate predictive models, and demonstrates that individuals dynamically adapt these in response to the predictability of their environment.

neuroscience↗

bioRxiv2023

Magnetic voluntary head-fixation in transgenic rats enables lifetime imaging of hippocampal neurons

The precise neural mechanisms within the brain that contribute to the remarkable lifetime persistence of memory remain unknown. Existing techniques to record neurons in animals are either unsuitable for longitudinal recording from the same cells or make it difficult for animals to express their full naturalistic behavioral repertoire. We present a magnetic voluntary head-fixation system that provides stable optical access to the brain during complex behavior. Compared to previous systems that used mechanical restraint, there are no moving parts and animals can engage and disengage entirely at will. This system is failsafe, easy for animals to use and reliable enough to allow long-term experiments to be routinely performed. Together with a novel two-photon fluorescence collection scheme that increases two-photon signal and a transgenic rat line that stably expresses the calcium sensor GCaMP6f in dorsal CA1, we are able to track and record activity from the same hippocampal neurons, during behavior, over a large fraction of animals lives.

neuroscience↗

bioRxiv2023

Learning generalizable visuomotor mappings for de novo skills

A fundamental feature of the human brain is its capacity to learn novel motor skills. This capacity requires the formation of vastly different visuomotor mappings. In this work, we ask how these associations are formed de novo, hypothesizing that under specific training regimes generalizable mappings are more readily formed, while in others, local state-actions associations are favored. To test this, we studied learning in a simple navigation task where participants attempted to move a cursor between various start-target locations by pressing three keyboard keys. Importantly, the mapping between the keys and the direction of cursor movement was unknown to the participants. Experiments 1 and 2 show that participants who were trained to move between multiple start-target pairs had significantly greater generalization than participants trained to move between a single pair. Whereas Experiment 1 found significant generalization when start-targets were distal, Experiment 2 found similar generalization for proximal targets, which suggests that generalization differences are due to knowledge of the visuomotor mapping itself and not simply due to planning. To gain insight into the potential computational mechanisms underlying this capacity, we explored how a visuomotor mapping could be formed through a set of models that afforded construction of a generalizable mappings (model-based), local state-action associations (model-free), or a hybrid of both. Our modeling work suggested that without continued variability between start-target pairs during training, model-based processes eventually gave way to model-free processes. In Experiment 3, we sought to further test this shift in learning processes by exposing participants to initially high variability before settling into a condition of no variability over a long-period of training. We found that generalization performance remained intact after a prolonged period of no variability suggesting that the formation of visuomotor mappings might occur at an early stage of learning. Finally, in Experiment 4 we show that adding stochasticity in the mapping can also promote model-based learning of a visuomotor mapping, suggesting that the learning may unfold implicitly. Overall, these studies shed light on how humans could acquire visuomotor mappings in their lives through exposure to variability in their feedback.

animal behavior and cognition↗