Biology subjects

Feher da Silva, C.

Publications and source records attributed to Feher da Silva, C..

6 recordsLinked to original sources

bioRxiv2026

A Computational Toolkit for Designing and Analysing Repeated Binary Choice Experiments

Adaptive human behaviour depends on the ability to detect regularities and probabilistic structures within a noisy environment. Repeated binary choice tasks, in which individuals predict one of two possible options, have long served as a fundamental tool for investigating learning, reward processing, and decision-making under uncertainty. However, traditional analyses of these tasks often rely on coarse measures such as accuracy or mean responses, overlooking the temporal information contained in behavioural sequences. This paper introduces a mathematical and computational framework to improve the analysis of binary choice data. First, we employ higher-order Markov chains to generate sequences with controlled probabilistic dependencies, allowing for a more subtle examination of how participants extract information and learn temporal structures. Presenting analytical methods derived from time series analysis (autocorrelation, cross---correlation, shifting probabilities and Markov reconstruction) to extract structural information from simulated data of prototypical behaviours, we demonstrate how these methods can identify distinct decisional patterns that remain hidden when using conventional approaches. Finally, we validate the proposed toolkit by applying it to empirical datasets. By revealing subtle but meaningful features of sequential decision-making, this framework enhances the interpretive power of probabilistic learning experiments. It provides researchers with tools to more accurately describe behavioural dynamics and deepens our understanding of the cognitive processes governing adaptive decisions in both healthy and clinical populations.

animal behavior and cognition↗

bioRxiv2022

A new take on model-based and model-free influences on mental effort and striatal prediction errors

A standard assumption in neuroscience is that low-effort model-free learning is automatic and continuously employed, while more complex model-based strategies are only used when the rewards they generate are worth the additional effort. We present evidence refuting this assumption. First, we demonstrate flaws in previous reports of combined model-free and model-based reward prediction errors in the ventral striatum that likely led to spurious results. More appropriate analyses yield no evidence of a model-free prediction errors in this region. Second, we find that task instructions generating more correct model-based behaviour reduce rather than increase mental effort. This is inconsistent with cost-benefit arbitration between model-based and model-free strategies. Together, our data suggest that model-free learning may not be automatic. Instead, humans can reduce mental effort by using a model-based strategy alone rather than arbitrating between multiple strategies. Our results call for re-evaluation of the assumptions in influential theories of learning and decision-making.

neuroscience↗

bioRxiv2019

Model-free or muddled models in the two-stage task?

Distinct model-free and model-based learning processes are thought to drive both typical and dysfunctional behaviours. Data from two-stage decision tasks have seemingly shown that human behaviour is driven by both processes operating in parallel. However, in this study, we show that more detailed task instructions lead participants to make primarily model-based choices that have little, if any, simple model-free influence. We also demonstrate that behaviour in the two-stage task may falsely appear to be driven by a combination of simple model-free and model-based learning if purely model-based agents form inaccurate models of the task because of misconceptions. Furthermore, we report evidence that many participants do misconceive the task in important ways. Overall, we argue that humans formulate a wide variety of learning models. Consequently, the simple dichotomy of model-free versus model-based learning is inadequate to explain behaviour in the two-stage task and connections between reward learning, habit formation, and compulsivity.

neuroscience↗

bioRxiv2017

Analysis of two-stage task results: how changes in task structure affect what model-free and model-based strategies predict about the effects of reward and transition on the stay probability

Many studies that aim to detect model-free and model-based influences on behavior employ two-stage behavioral tasks of the type pioneered by Daw and colleagues in 2011. Such studies commonly modify existing two-stage decision paradigms in order to better address a given hypothesis, which is an important means of scientific progress. It is, however, critical to fully appreciate the impact of any modified or novel experimental design features on the expected results. Here, we use two concrete examples to demonstrate that relatively small changes in the two-stage task design can substantially change the pattern of actions taken by model-free and model-based agents as a function of the reward outcomes and transitions on previous trials. In the first, we show that, under specific conditions, purely model-free agents will produce the reward by transition interactions typically thought to characterize model-based behavior on a two-stage task. The second example shows that model-based agents behavior is driven by a main effect of transition-type in addition to the canonical reward by transition interaction whenever the reward probabilities of the final states do not sum to one. Together, these examples emphasize the task-dependence of model-free and model-based behavior and highlight the benefits of using computer simulations to determine what pattern of results to expect from both model-free and model-based agents performing a given two-stage decision task in order to design choice paradigms and analysis strategies best suited to the current question.

neuroscience↗

bioRxiv2017

Model-free reinforcement learning operates over information stored in working-memory to drive human choices

Model-free learning creates stimulus-response associations. But what constitutes a stimulus? Are there limits to types of stimuli a model-free or habitual system can operate over? Most experiments on reward learning in humans and animals have used discrete sensory stimuli, but there is no algorithmic reason that model-free learning should be restricted to external stimuli, and recent theories have suggested that model-free processes may operate over highly abstract concepts and goals. Our study aimed to determine whether model-free learning processes can operate over environmental states defined by information held in working memory. Specifically, we tested whether or not humans can learn explicit temporal patterns of individually uninformative cues in a model-free manner. We compared the data from human participants in a reward learning paradigm using (1) a simultaneous symbol presentation condition or (2) a sequential symbol presentation condition, wherein the same visual stimuli were presented simultaneously or as a temporal sequence that required working memory. We found a significant effect of reward on human behavior in the sequential presentation condition, indicating that model-free learning can operate on information stored in working memory. Further analyses, however, revealed that the behavior of the participants contradicts the basic assumptions of our hypotheses, and it is possible that the observed effect of reward was generated by model-based rather than model-free learning. Thus it is not possible to draw any conclusions from out study regarding model-free learning of temporal sequences held in working memory. We conclude instead that careful thought should be given about how to best explain two-stage tasks to participants.

neuroscience↗

bioRxiv2017

Exploration and recency as the main proximate causes of probability matching: a reinforcement learning analysis

Research has not yet reached a consensus on why humans match probabilities instead of maximise in a probability learning task. The most influential explanation is that they search for patterns in the random sequence of outcomes. Other explanations, such as expectation matching, are plausible, but do not consider how reinforcement learning shapes peoples choices.\n\nWe aimed to quantify how human performance in a probability learning task is affected by pattern search and reinforcement learning. We collected behavioural data from 84 young adult participants who performed a probability learning task wherein the majority outcome was rewarded with 0.7 probability, and analysed the data using a reinforcement learning model that searches for patterns. Model simulations indicated that pattern search, exploration, recency (discounting early experiences), and forgetting may impair performance.\n\nOur analysis estimated that 85% (95% HDI [76, 94]) of participants searched for patterns and believed that each trial outcome depended on one or two previous ones. The estimated impact of pattern search on performance was, however, only 6%, while those of exploration and recency were 19% and 13% respectively. This suggests that probability matching is caused by uncertainty about how outcomes are generated, which leads to pattern search, exploration, and recency.

animal behavior and cognition↗