Biology subjects

Khamassi, M.

Publications and source records attributed to Khamassi, M..

13 recordsLinked to original sources

bioRxiv2026

Sensorimotor encoding of epistemic value during goal-directed causal learning

Understanding the neural and computational mechanisms underlying goal-directed causal learning is a central challenge in both cognitive neuroscience and artificial intelligence. This cognitive function depends on balancing reward maximization with information seeking. Although substantial progress has been made in characterizing the neural basis of reward-driven learning, it remains unclear whether and how intrinsic informational value is represented in the brain and propagated through cortico-cortical interactions. Here, we dissociate information-seeking from reward maximisation during goal-directed causal learning using a novel behavioural paradigm in which participants estimate action-outcome contingencies without extrinsic incentives. Bayesian computational modelling reveals that while random exploration dominates behaviour, expected information gain (EIG) explains a significant proportion of exploratory choices in a subset of sessions. Using magnetoencephalography combined with information-theoretic analyses of high-gamma activity, we show that EIG is encoded in the left sensorimotor cortex at the time of action execution, specifically in dorsal premotor and primary motor cortex, and broadcast to primary somatosensory cortex. These findings are consistent with Active Inference predictions that EIG constitutes a key computational drive for exploratory action selection. However, they challenge the view that epistemic valuation exclusively recruits prefrontal reward circuits, and instead they support an embodied account in which the premotor and sensorimotor system mediates the intrinsic valuation during first-personal interventional causal learning.

neuroscience↗

bioRxiv2025

A model-centered introduction to curiosity

AO_SCPLOWBSTRACTC_SCPLOWCuriosity, defined as a transient motivational state for information, explains many non-greedy decisions in humans. Novel, surprising and uncertain situations are thought to elicit curiosity, leading to various models for information-seeking behaviors. We present a unified comparison of curiosity models based on experimental results and previous models from the decision-making and reinforcement learning literature. We provide and explain at least one mathematical formula for each curiosity category, implement them, and compare them on the same uncertain and changing task. This comparison illustrates how motivation and exploration may evolve differently based on what elicits curiosity. This work promises to increase the understanding of curiosity models and provide a synthesized toolbox for future models of information-seeking behaviors in animals or artificial agents.

animal behavior and cognition↗

bioRxiv2025

The hippocampus as an epistemic forager: When curiosity and reward jointly steer exploration and hippocampal replay

Hippocampal replay is a widely studied phenomenon wherein special neurons of the hippocampus encoding spacial locations - place cells - show a sequential reactivation during periods of immobility, often representing trajectories to or from reward locations, as observed in foraging rodents. Several computational models have been proposed to explain how this phenomenon could contribute to memory consolidation and action planning. However, certain aspects of the mechanism behind hippocampal replay remain unclear, such as why reactivation is biased towards both reward sites and decision points. Here, we propose that both expected reward (satisfying hunger) and expected information gain (satisfying curiosity) contribute to determine the priority of events to be replayed. To test this, we present the Epistemic Replay Algorithm (ERA), which bridges reinforcement learning and active inference into a single computational model. We evaluate the ERA in five experiments spanning three maze types: linear maze, non-stationary maze, double T-maze. Our results first showcase that more curious agents explore more thoroughly while they are still capable of exploiting optimal rewards; and they can adapt faster to changing environments. Further, we find that the ERA model accounts for a larger number of hippocampal replay properties compared to non-curious models, including (i) a broad-to-specific progression of hippocampal replay events; (ii) symmetric replay around decision points; and (iii) the preferential reactivation of both reward sites and decision points. We derive new predictions to further test the model and discuss its implications compared to alternative accounts.

neuroscience↗

bioRxiv2025

Multi-model reinforcement learning with online retrospective change-point detection

AO_SCPLOWBSTRACTC_SCPLOWHumans continuously adapt to uncertain and changing situations. However, most reinforcement learning models of human behavior struggle to explain this capability. We propose a novel reinforcement learning agent for uncertain and volatile Markov decision processes, which we call Multi-Model with Retrospective Change Point Detection (MMRCPD). MMRCPD relies on two novel ideas: arbitrating between local models rather than contexts of the environment and retrospectively detecting change points. Arbitrating between local models limits memory costs and enables faster adaptation to new contexts which sub-parts have been experienced before. Retrospective change point detection mimics the capacity of humans to infer the latent cause of a change after it happened and maintain precise models of the environment. MMRCPD can detect local changes online, create new models, retrospectively update its models based on when it estimates that the change happened, reuse past models, merge models if they become similar, and forget unused models. This novel multi-model agent outperforms single-model and context-level change-detection methods in uncertain and locally changing environments. These results yield new insights and predictions concerning optimal decision-making in changing and uncertain environments, which could in turn be tested in behavioral experiments.

animal behavior and cognition↗

bioRxiv2025

Recasting adaptation as strategy inference

Flexible adaptation to uncertain and changing environments requires dynamic adjustments in behavioral strategies. While classical learning theories emphasize incremental strengthening of local stimulus-action associations in adaptation, emerging evidence suggests that global-level strategy representations may enable rapid inference of adaptive behaviors, thus promoting efficient decision-making. However, it remains unclear to what extent direct inference over putative strategies can fully account for human adaptation across diverse statistical contexts. Here, we demonstrate clear behavioral markers supporting the broad use of inference over strategies in human adapting to rapid changes. These markers are fully explained solely by a novel model of inference over a structured space of strategies. We further show that inference over strategies is influenced by latent contextual statistics that are beyond the scope of models based on incremental learning. Taken together, these results establish the importance of direct inference over an abstract strategy space for flexible adaptation in humans.

animal behavior and cognition↗

bioRxiv2024

Rat anterior cingulate neurons responsive to rule or strategy changes are modulated by the hippocampal theta rhythm and sharp-wave ripples

To better understand neural processing during adaptive learning of stimulus-response-reward contingencies, we recorded synchrony of neuronal activity in anterior cingulate cortex (ACC) with hippocampal rhythms in male rats acquiring and switching between spatial and visual discrimination tasks in a Y-maze. ACC population and single unit activity responded shortly after task rule changes, or just before the rats adopted different task strategies. Hippocampal theta oscillations (associated with memory encoding) modulated an elevated proportion of rule-change responsive neurons (70%), but other neurons that were correlated with strategy-change, strategy value, and reward-rate were not. However, hippocampal sharp wave-ripples modulated significantly higher proportions of rule-change, strategy-change and reward-rate responsive cells during post-session sleep but not pre-session sleep. This suggests an underestimated mechanism for hippocampal mismatch and contextual signals to facilitate ACC detection of contingency changes for cognitive flexibility, a function that is attenuated after it is damaged.

neuroscience↗

bioRxiv2022

Learning Fast and Slow: Increased cortical plasticity leads to memory interference and enhanced hippocampal-cortical interactions

Our brain is continuously challenged by daily experiences. Thus, how to avoid systematic erasing of previously encoded memories? While it has been proposed that a dual-learning system with "slow" learning in the cortex and "fast" learning in the hippocampus could protect previous knowledge from interference, this has never been observed in the living organism. Here, we report that increasing plasticity via the viral-induced overexpression of RGS14414 in the prelimbic cortex leads to better one-trial memory, but that this comes at the price of increased interference in semantic-like memory. Indeed, electrophysiological recordings showed that this manipulation also resulted in shorter NonREM-sleep bouts, smaller delta-waves and decreased neuronal firing rates. In contrast, hippocampal-cortical interactions in form of theta coherence during wake and REM-sleep as well as oscillatory coupling during NonREM-sleep were enhanced. Thus, we provide the first experimental evidence for the long-standing and unproven fundamental idea that high thresholds for plasticity in the cortex protects preexisting memories and modulating these thresholds affects both memory encoding and consolidation mechanisms.

neuroscience↗

bioRxiv2021

Social prediction modulates activity of macaque superior temporal cortex

The ability to attribute thoughts to others, also called theory of mind (TOM), has been extensively studied. Computationally, the basis of TOM in humans has been interpreted within the predictive coding framework and associated with activity in the temporo-parietal junction (TPJ). However, the evolutionary origins of these human mindreading abilities have been challenged since the concept was coined. Here we identify a brain region in the Rhesus macaque that shares computational properties with the human TPJ. We revealed, using a non-linguistic task and functional magnetic resonance imaging, that activity in a region of the macaque middle superior temporal cortex was specifically modulated by the predictability of social interactions. As in human TPJ, this region could be distinguished from other temporal regions involved in face processing. Our result suggests the existence of a precursor for the theory of mind ability in the last common ancestor of human and old-world monkeys.

neuroscience↗

bioRxiv2019

The Object Space Task reveals a dissociation between semantic-like and episodic-like memory in a mouse model of Kleefstra Syndrome

Kleefstra syndrome is a disorder caused by a mutation in the EHMT1 gene characterized in humans by general developmental delay, mild to severe intellectual disability and autism. Here, we characterized semantic- and episodic-like memory in the Ehmt1+/- mouse model using the Object Space Task. We combined conventional behavioral analysis with automated analysis by deep-learning networks, a session-based computational learning model and a trial-based classifier. Ehmt1 +/- mice showed more anxiety-like features and generally explored objects less, but the difference decreased over time. Interestingly, when analyzing memory-specific exploration, Ehmt1 +/- show increased expression of semantic-like memory, but a deficit in episodic-like memory. A similar dissociation of semantic and episodic memory performance has been previously reported in humans with autism. Using our automatic classifier to differentiate between genotypes, we found that semantic-like memory features are better suited for classification than general exploration differences. Thus, detailed behavioral classification with the Object Space Task produced a more detailed behavioral phenotype of the Ehmt1 +/- mouse model.\n\nOne Sentence SummaryEhmt1 +/- mice show decreased exploration and episodic-like memory but increased semantic-like memory In the Object Space Task. (143 of 150)

neuroscience↗

bioRxiv2018

Dopamine regulates the exploration-exploitation trade-off in rats

In a volatile environment where rewards are uncertain, successful performance requires a delicate balance between exploitation of the best option and exploration of alternative choices. It has theoretically been proposed that dopamine controls this exploration-exploitation trade-off, specifically that the higher the level of tonic dopamine, the more exploitation is favored. We demonstrate here that there is a formal relationship between the rescaling of dopamine positive reward prediction errors and the exploration-exploitation trade-off in simple non-stationary multi-armed bandit tasks. We further show in rats performing such a task that systemically antagonizing dopamine receptors greatly increases the number of random choices without affecting learning capacities. Simulations and comparison of a set of different computational models (an extended Q-learning model, a directed exploration model, and a meta-learning model) fitted on each individual confirm that, independently of the model, decreasing dopaminergic activity does not affect learning rate but is equivalent to an increase in exploration rate. This study shows that dopamine could adapt the exploration-exploitation trade-off in decision making when facing changing environmental contingencies.

neuroscience↗

bioRxiv2018

The computation of strategic learning in repeated social competitive interactions: Learning sophistication, reward attractor points and strategic asymmetry

Social interactions rely on our ability to learn and adjust our behavior to the behavior of others. Strategic games provide a useful framework to study the cognitive processes involved in the formation of beliefs about the others intentions and behavior, what we may call strategic theory of mind. Through the years, the growing field of behavioral economics provided evidence of a systematic departure of humans behavior from the optimal game theoretical prescriptions. One hypothesis posits that humans ability to accurately process the others behavior is somehow bounded. The question of what constraints the formation of sufficiently high order beliefs remained unanswered. We hypothesize that maximizing final earnings in a competitive repeated game setting, requires moving away from reward-based learning to engage in sophisticated belief-based learning. Overcoming the attraction of the immediate rewards by displaying a computationally costly type of learning might not be a strategy shared among all individuals. In this work, we manipulated the reward structure of the interaction so that the action displayed by the two types of learning becomes (respectively not) discriminable, giving a relative strategic (resp. dis) advantage to the participant given the role endorsed during the interaction. We employed a computational modeling approach to characterize the individual level of belief learning sophistication in three types of interactions (agent-agent, human-human and human-agent). The analysis of the participants choice behavior revealed that the strategic learning level drives the formation of more accurate beliefs and eventually leads to convergence towards game optimality (equilibrium). More specifically we show that the game structure interacts with the level of engagement in strategically sophisticated learning to explain the outcome of the interaction. This study provides the first evidence of a key implication of strategic learning heterogeneity in equilibrium departure and provides insight to explain the emergence of a leader-follower dynamics of choice.\n\nAUTHOR SUMMARYDynamic interaction between individuals appears to be a cornerstone for understanding how humans grasp other minds. During a strategic interaction, in which the outcome of ones action depends directly on what the other individual decides, it appears crucial to anticipate the others actions in order to adjust our own behavior. In theory, choosing optimally in a strategic setting requires that both players hold correct beliefs over their opponents behavior and best-respond to it. However, in practice humans systematically deviate from the game-theoretical (equilibrium), suggesting that our ability to form accurate beliefs is cognitively and/or contextually constrained. Previous studies using computational modelling suggested that during a repeated game interaction humans vary in the sophistication of their learning process leading to the formation of beliefs over their opponents behavior of different orders of complexity (level of recursive thinking such as \"I think that you think that ...\"). In this work we show that the individual engagement in sophisticated (belief-based) learning drives the convergence towards equilibrium and ultimately performance. Moreover, we show that this effect is influenced by both the game environment and the cognitive capacity of the participants, shaping the very dynamic of the social interaction.\n\nDATA AVAILABILITYThe authors confirm that upon publication the raw behavioral data and Matlab code for reconstruction of all figures, computational models and statistical analyses will be made available for download at the following URL: https://zenodo.org/

neuroscience↗

bioRxiv2018

Reference-point centering and range-adaptation enhance human reinforcement learning at the cost of irrational preferences

In economics and in perceptual decision-making contextual effects are well documented, where decision weights are adjusted as a function of the distribution of stimuli. Yet, in reinforcement learning literature whether and how contextual information pertaining to decision states is integrated in learning algorithms has received comparably little attention. Here, in an attempt to fill this gap, we investigated reinforcement learning behavior and its computational substrates in a task where we orthogonally manipulated both outcome valence and magnitude, resulting in systematic variations in state-values. Over two experiments, model comparison indicated that subjects behavior is best accounted for by an algorithm which includes both reference point-dependence and range-adaptation - two crucial features of state-dependent valuation. In addition, we found state-dependent outcome valuation to progressively emerge over time, to be favored by increasing outcome information and to be correlated with explicit understanding of the task structure. Finally, our data clearly show that, while being locally adaptive (for instance in negative valence and small magnitude contexts), state-dependent valuation comes at the cost of seemingly irrational choices, when options are extrapolated out from their original contexts.

animal behavior and cognition↗

bioRxiv2017

Bridging Computational Neuroscience and Machine Learning on Non-Stationary Multi-Armed Bandits

Fast adaptation to changes in the environment requires both natural and artificial agents to be able to dynamically tune an exploration-exploitation trade-off during learning. This trade-off usually determines a fixed proportion of exploitative choices (i.e. choice of the action that subjectively appears as best at a given moment) relative to exploratory choices (i.e. testing other actions that now appear worst but may turn out promising later). The problem of finding an efficient exploration-exploitation trade-off has been well studied both in the Machine Learning and Computational Neuroscience fields. Rather than using a fixed proportion, non-stationary multi-armed bandit methods in the former have proven that principles such as exploring actions that have not been tested for a long time can lead to performance closer to optimal - bounded regret. In parallel, researches in the latter have investigated solutions such as progressively increasing exploitation in response to improvements of performance, transiently increasing exploration in response to drops in average performance, or attributing exploration bonuses specifically to actions associated with high uncertainty in order to gain information when performing these actions. In this work, we first try to bridge some of these different methods from the two research fields by rewriting their decision process with a common formalism. We then show numerical simulations of a hybrid algorithm combining bio-inspired meta-learning, kalman filter and exploration bonuses compared to several state-of-the-art alternatives on a set of non-stationary stochastic multi-armed bandit tasks. While we find that different methods are appropriate in different scenarios, the hybrid algorithm displays a good combination of advantages from different methods and outperforms these methods in the studied scenarios.

animal behavior and cognition↗