Biology subjects

Chartouny, A.

Publications and source records attributed to Chartouny, A..

3 recordsLinked to original sources

bioRxiv2025

A model-centered introduction to curiosity

AO_SCPLOWBSTRACTC_SCPLOWCuriosity, defined as a transient motivational state for information, explains many non-greedy decisions in humans. Novel, surprising and uncertain situations are thought to elicit curiosity, leading to various models for information-seeking behaviors. We present a unified comparison of curiosity models based on experimental results and previous models from the decision-making and reinforcement learning literature. We provide and explain at least one mathematical formula for each curiosity category, implement them, and compare them on the same uncertain and changing task. This comparison illustrates how motivation and exploration may evolve differently based on what elicits curiosity. This work promises to increase the understanding of curiosity models and provide a synthesized toolbox for future models of information-seeking behaviors in animals or artificial agents.

animal behavior and cognition↗

bioRxiv2025

The hippocampus as an epistemic forager: When curiosity and reward jointly steer exploration and hippocampal replay

Hippocampal replay is a widely studied phenomenon wherein special neurons of the hippocampus encoding spacial locations - place cells - show a sequential reactivation during periods of immobility, often representing trajectories to or from reward locations, as observed in foraging rodents. Several computational models have been proposed to explain how this phenomenon could contribute to memory consolidation and action planning. However, certain aspects of the mechanism behind hippocampal replay remain unclear, such as why reactivation is biased towards both reward sites and decision points. Here, we propose that both expected reward (satisfying hunger) and expected information gain (satisfying curiosity) contribute to determine the priority of events to be replayed. To test this, we present the Epistemic Replay Algorithm (ERA), which bridges reinforcement learning and active inference into a single computational model. We evaluate the ERA in five experiments spanning three maze types: linear maze, non-stationary maze, double T-maze. Our results first showcase that more curious agents explore more thoroughly while they are still capable of exploiting optimal rewards; and they can adapt faster to changing environments. Further, we find that the ERA model accounts for a larger number of hippocampal replay properties compared to non-curious models, including (i) a broad-to-specific progression of hippocampal replay events; (ii) symmetric replay around decision points; and (iii) the preferential reactivation of both reward sites and decision points. We derive new predictions to further test the model and discuss its implications compared to alternative accounts.

neuroscience↗

bioRxiv2025

Multi-model reinforcement learning with online retrospective change-point detection

AO_SCPLOWBSTRACTC_SCPLOWHumans continuously adapt to uncertain and changing situations. However, most reinforcement learning models of human behavior struggle to explain this capability. We propose a novel reinforcement learning agent for uncertain and volatile Markov decision processes, which we call Multi-Model with Retrospective Change Point Detection (MMRCPD). MMRCPD relies on two novel ideas: arbitrating between local models rather than contexts of the environment and retrospectively detecting change points. Arbitrating between local models limits memory costs and enables faster adaptation to new contexts which sub-parts have been experienced before. Retrospective change point detection mimics the capacity of humans to infer the latent cause of a change after it happened and maintain precise models of the environment. MMRCPD can detect local changes online, create new models, retrospectively update its models based on when it estimates that the change happened, reuse past models, merge models if they become similar, and forget unused models. This novel multi-model agent outperforms single-model and context-level change-detection methods in uncertain and locally changing environments. These results yield new insights and predictions concerning optimal decision-making in changing and uncertain environments, which could in turn be tested in behavioral experiments.

animal behavior and cognition↗