Biology subjects

Kiebel, S. J.

Publications and source records attributed to Kiebel, S. J..

14 recordsLinked to original sources

bioRxiv2026

Task-specific preferences guide forward planning in sequential decision-making

Everyday problems often require choosing among actions with different future consequences, even though fully evaluating those consequences is often computationally infeasible. The human brain is remarkable in its ability to solve such complex problems efficiently. In this paper, we hypothesize that this capacity arises from a principle inspired by Bayesian accounts of perception and sensorimotor control: evidence derived from effortful forward planning is integrated with task-specific preferences akin to priors, weighted by the uncertainty of these preferences. To test this hypothesis, we used a sequential decision-making task with a relatively large state space. Model comparison showed that participants' deviations from optimal forward planning are best captured by task-specific preferences, rather than by unsystematic errors, recent action-history effects, or offer-context biases. Moreover, the influence of forward planning on participants' responses depended on preference uncertainty, as evidenced by both choice behavior and reaction times, and preferences were not arbitrary but instead supported above-stochastic performance even in the absence of forward planning. Together, these results are consistent with a Bayesian-inspired interpretation of decision making, in which costly forward planning is integrated with state-dependent priors or preferences. These preferences may provide an inexpensive, quickly available first-pass solution, which is complemented by forward planning when preference-based guidance is uncertain.

neuroscience↗

bioRxiv2026

Reduced planning depth and stronger probability discounting in alcohol and tobacco use disorder

Background: Substance use disorders (SUDs) are associated with impaired goal-directed decision-making, but the specific cognitive mechanisms underlying these deficits remain unclear. Forward planning, a key component of goal-directed control, requires strategies to contain computational demand and ensure feasibility through heuristics such as reducing planning depth, pruning unlikely options, and discounting probabilistic outcomes. This study investigated whether individuals with alcohol use disorder (AUD) and tobacco use disorder (TUD) differ from healthy controls in their use of these heuristics during sequential decision-making. Methods: We examined participants from the general population meeting criteria for at least moderate AUD (n = 45), TUD (n = 49), or both (n = 45), and compared them with matched healthy control groups. To assess forward planning, we used a previously published probabilistic three-step Markov decision task. Computational modeling was used to infer individual planning depths and compare model variants incorporating low-probability pruning, probability discounting, or both. Results: A combined model including both low-probability pruning and probability discounting best explained participants' behavior and was subsequently used for analysis. Compared to matched healthy controls, all groups with SUD exhibited significantly reduced planning depth. Moreover, individuals with SUD tended to show stronger probability discounting, with this effect reaching statistical significance in the comorbid group. Conclusion: Individuals with AUD and/or TUD engaged in forward planning using similar strategies as healthy controls but exhibited reduced planning depth and a tendency toward stronger probability discounting. These findings suggest the presence of altered planning processes in SUD, which could contribute to maladaptive decision-making. Future research is needed to clarify whether such differences reflect pre-existing cognitive traits, consequences of substance use, or both, and to explore their potential relevance for intervention.

neuroscience↗

bioRxiv2024

The role of repetition in context-dependent preference

Humans are prone to decision biases, which make behavior seemingly irrational. An important cause for decision biases is that the context in which decisions are made can later influence which choices humans prefer in new situations. Current computational models often require extensive environmental knowledge to explain these biases. Here, we tested the hypothesis that decision biases are mainly driven by a tendency to repeat context-specific actions. We implemented a series of nine value-based decision experiments on n = 351 male and female participants and reanalyzed six previously published datasets (n=350 participants). We found that higher within-context repetition of an option was associated with biased choices including higher subjective valuation and lower uncertainty for repeated actions. Next, we used a computational model based on two basic principles, learning by reward and decision repetition and tested it on all datasets. Our results show that the combination of these two basic principles is sufficient to explain biased choices. We demonstrate via hierarchical Bayesian model comparison that our model outperforms all alternative models. These results provide novel insights into human choice behavior and offer a solution to difficulties in current explanations and models. Significance StatementHuman decision-making often deviates from rational choice theory. Understanding how decision biases emerge is essential for interpreting real-world human behavior. Our study shows how repeating decisions shapes decision making in novel situations. We adapt a simple computational model that can explain decision biases based on reward learning and decision repetition. By testing the model across 15 datasets, we demonstrate that it outperforms previous models, also offering new insights into how preferences affect subjective valuation and uncertainty. In sum, these findings provide a deeper understanding of how context-dependent preferences for specific decisions emerge and persist.

neuroscience↗

bioRxiv2024

Context-Dependent Interaction Between Goal-Directed and Habitual Control Under Time Pressure

Habits are an important aspect of human behaviour. Habits are reflexive, inflexible, and fast, in contrast to goal-directed behaviour which is reflective, flexible, and slow. Current theories assume that habits and goal-directed actions are controlled by two separate but interacting systems. However, it is not clear how these two systems interact when actions must be made under time pressure. Here we use a task which induces habitual behaviour in the form of action sequences, while concurrently requiring participants to perform goal-directed actions that are either congruent or incongruent with the habit. This task thus allows for concurrent measurement of both goal-directed and habitual behaviour, thereby permitting a nuanced analysis of the interaction between these two control modes. Using computational modelling, we find that models where the influence of the habit depends on the number of repetitions, explain participant behaviour better than models that assume the habit to be constant. We further show that roughly half of the participants modulate their use of the habit depending on the context, i.e. they selectively inhibit the habits influence when it is incongruent to their explicit goals, but not when both are congruent and the influence of the habit is adaptive. Additional drift-diffusion modelling of choice and reaction time data shows that proactive control is mobilized in the congruent task context whereas reactive control is mobilized in the incongruent task context. The present study thus indicates that habitual control is context-dependent and can be adaptively deployed via proactive and reactive control, rather than being a fixed or isolated mechanism.

neuroscience↗

bioRxiv2024

Cognitive Computational Model Reveals Repetition Bias in a Sequential Decision-Making Task

Humans tend to repeat past actions due to rewarding outcomes. Recent computational models propose that the probability of selecting a specific action is also, in part, based on how often this action was selected before, independent of previous outcomes or reward. However, these new models so far lack empirical support. Here, we present evidence of a repetition bias using a novel sequential decision-making task and computational modeling to reveal the influence of choice frequency on human value-based choices. Specifically, we find that value-based decisions can be best explained by concurrent influence of both goal-directed reward seeking and a repetition bias. We also show that participants differ substantially in their repetition bias strength, and relate these measures to task performance. The new task enables a novel way to measure the influence of choice repetition on decision-making. These findings can serve as a basis for further experimental studies on the interplay between rewards and choice history in human decision-making.

neuroscience↗

bioRxiv2022

Forward planning under uncertainty in a population-based alcohol use disorder sample

BackgroundEtiological theories of addictive behavior postulate a key role for decision-making mechanisms. However, current research is lacking clear computational models for decision-making in multi-step forward planning scenarios to identify underlying mechanisms and make derived hypotheses testable. MethodsWe used a recently developed planning task and computational model to investigate performance, planning time and inferred forward planning parameters like planning depth and decision noise in 30 individuals diagnosed with mostly mild-to-moderate alcohol use disorder (AUD) relative to 32 healthy control participants, both sampled from the general population. ResultsContrary to our hypothesis, we did not observe reduced planning depth in participants with AUD but found that participants with AUD showed a higher performance in the planning task. Group differences could be explained by planning time and general cognitive performance. Importantly, participants with AUD invested more time for planning, showed a higher correlation of planning depth with incentive value and showed lower response noise, potentially indicative of higher choice consistency. ConclusionThe significant differences in planning time, moderation of planning depth by incentive value and choice consistency may reflect higher motivation and willingness to exert effort among participants with AUD compared to healthy controls. Overall, our findings do not support the notion that mild-to-moderate alcohol use disorder is associated with impairments in forward planning across multiple steps.

neuroscience↗

bioRxiv2022

Interaction of automatic motor and goal-directed behavior under time pressure

Human behaviour consists in large parts of action sequences that are often repeated in mostly the same way. Through extensive repetition, sequential responses become automatic or habitual, but our environment often confronts us with events to which we have to react flexibly and in a goal-directed manner. To assess how implicitly learned action sequences interfere with goal-directed control, we developed a novel behavioural paradigm in which we combined action sequence learning through repetition with a goal-directed task component. So-called dual-target trials require the goal-directed selection of the response with the highest reward-probability in a fast succession of trials with short response deadlines. Importantly, the response primed by the learned action sequence is sometimes different from that required by the goal-directed task. As expected, we found that participants learned the action sequence through repetition, as evidenced by reduced reaction times and error rates, while still acting in a goal-directed manner in dual target trials. Specifically, we found that the learned action sequence biased choices in the goal-directed task towards the sequential response, and this effect was more pronounced the better individuals had learned the sequence. Our novel task may help shed light on the acquisition of automatic behavioural patterns and habits through extensive repetition, allows to assess positive features of habitual behaviour (e.g. increased response speed and reduced error rates), and importantly also the interaction of habitual and goal-directed behaviours under time pressure.

animal behavior and cognition↗

bioRxiv2022

Revealing human sensitivity to a latent temporal structure of changes

Precisely timed behaviour and accurate time perception plays a critical role in our everyday lives, as our well-being and even survival can depend on well-timed decisions. Although the temporal structure of the world around us is essential for human decision making, we know surprisingly little about how representation of temporal structure of our everyday environment impacts decision making. How does the representation of temporal structure affect our ability to generate well-timed decisions? Here we address this question by using a well-established dynamic probabilistic learning task. Using computational modelling, we found that human subjects beliefs about temporal structure are reflected in their choices to either exploit their current knowledge or to explore novel options. The model-based analysis illustrates a large within-group and within-subject heterogeneity. To explain these results, we propose a normative model for how temporal structure is used in decision making, based on the semi-Markov formalism in an active inference framework. We discuss potential key applications of the presented approach to the fields of cognitive phenotyping and computational psychiatry.

neuroscience↗

bioRxiv2021

The effects of uncertain context inference on motor adaptation

Humans have been shown to adapt their movements when a sudden or gradual change to the dynamics of the environment are introduced, a phenomenon called motor adaptation. If the change is reverted, the adaptation is also quickly reverted. Humans are also able to adapt to multiple changes in dynamics presented separately, and to be able to switch between adapted movements on the fly. Such switching relies on contextual information which is often noisy or misleading, affecting the switch between known adaptations. Recently, computational models for motor adaptation and context inference have been introduced, which contain components for context inference and Bayesian motor adaptation. These models were used to show the effects of context inference on learning rates across different experiments. We expanded on these works by using a simplified version of the recently-introduced COIN model to show that the effects of context inference on motor adaptation and control go even further than previously shown. Here, we used this model to simulate classical motor adaptation experiments from previous works and showed that context inference, and how it is affected by the presence and reliability of feedback, effect a host of behavioral phenomena that had so far required multiple hypothesized mechanisms, lacking a unified explanation. Concretely, we show that the reliability of direct contextual information, as well as noisy sensory feedback, typical of many experiments, effect measurable changes in switching-task behavior, as well as in action selection, that stem directly from probabilistic context inference.

neuroscience↗

bioRxiv2021

Network instability dynamics drive a transient bursting period in the developing hippocampus in vivo

Spontaneous correlated activity is a universal hallmark of immature neural circuits. However, the cellular dynamics and intrinsic mechanisms underlying network burstiness in the intact developing brain are largely unknown. Here, we use two-photon Ca2+ imaging to comprehensively map the developmental trajectories of spontaneous network activity in hippocampal area CA1 in vivo. We unexpectedly find that network burstiness peaks after the developmental emergence of effective synaptic inhibition in the second postnatal week. We demonstrate that the enhanced network burstiness reflects an increased functional coupling of individual neurons to local population activity. However, pairwise neuronal correlations are low, and network bursts (NBs) recruit CA1 pyramidal cells in a virtually random manner. Using a dynamic systems modeling approach, we reconcile these experimental findings and identify network bi-stability as a potential regime underlying network burstiness at this age. Our analyses reveal an important role of synaptic input characteristics and network instability dynamics for NB generation. Collectively, our data suggest a mechanism, whereby developing CA1 performs extensive input-discrimination learning prior to the onset of environmental exploration.

neuroscience↗

bioRxiv2020

Intraneuronal chloride accumulation via NKCC1 is not essential for hippocampal network development in vivo

NKCC1 is the primary transporter mediating chloride uptake in immature principal neurons, but its role in the development of in vivo network dynamics and cognitive abilities remains unknown. Here, we address the function of NKCC1 in developing mice using electrophysiological, optical and behavioral approaches. We report that NKCC1 deletion from telencephalic glutamatergic neurons decreases in-vitro excitatory GABA actions and impairs neuronal synchrony in neonatal hippocampal brain slices. In vivo, it has a minor impact on correlated spontaneous activity in the hippocampus and does not affect network activity in the intact visual cortex. Moreover, long-term effects of the developmental NKCC1 deletion on synaptic maturation, network dynamics and behavioral performance are subtle. Our data reveal a neural network function of depolarizing GABA in the hippocampus in vivo, but challenge the hypothesis that NKCC1 is essential for major aspects of hippocampal development.

neuroscience↗

bioRxiv2019

Meta-control of the exploration-exploitation dilemma emerges from probabilistic inference over a hierarchy of time scales

Cognitive control is typically understood as a set of mechanisms which enable humans to reach goals that require integrating the consequences of actions over longer time scales. Importantly, using routine beheavior or making choices beneficial only at a short time scales would prevent one from attaining these goals. During the past two decades, researchers have proposed various computational cognitive models that successfully account for behaviour related to cognitive control in a wide range of laboratory tasks. As humans operate in a dynamic and uncertain environment, making elaborate plans and integrating experience over multiple time scales is computationally expensive, the specific question of how uncertain consequences at different time scales are integrated into adaptive decisions remains poorly understood. Here, we propose that precisely the problem of integrating experience and forming elaborate plans over multiple time scales is a key component for better understanding how human agents solve cognitive control dilemmas such as the exploration-exploitation dilemma. In support of this conjecture, we present a computational model of probabilistic inference over hidden states and actions, which are represented as a hierarchy of time scales. Simulations of goal-reaching agents instantiating the model in an uncertain and dynamic task environment show how the exploration-exploitation dilemma may be solved by inferring meta-control states which adapt behaviour to changing contexts.

neuroscience↗

bioRxiv2019

Dynamic integration of forward planning and heuristic preferences during multiple goal pursuit

Selecting goals and successfully pursuing them in an uncertain and dynamic environment is an important aspect of human behaviour. In order to decide which goal to pursue at what point in time, one has to evaluate the consequences of ones actions over future time steps by forward planning. However, when the goal is still temporally distant, detailed forward planning can be prohibitively costly. One way to select actions at minimal computational costs is to use heuristics. It is an open question how humans mix heuristics with forward planning to balance computational costs with goal reaching performance. To test a hypothesis about dynamic mixing of heuristics with forward planning, we used a novel stochastic sequential two-goal task. Comparing participants decisions with an optimal full planning agent, we found that at the early stages of goal-reaching sequences, in which both goals are temporally distant and planning complexity is high, on average 42% (SD = 19%) of participants choices deviated from the agents optimal choices. Only towards the end of the sequence, participants behaviour converged to near optimal performance. Subsequent model-based analyses showed that participants used heuristic preferences when the goal was temporally distant and switched to forward planning when the goal was close. Author summaryWhen we pursue our goals, there is often a moment when we recognize that we did not make the progress that we hoped for. What should we do now? Persevere to achieve the original goal, or switch to another goal? Two features of real-world goal pursuit make these decisions particularly complex. First, goals can lie far into an unpredictable future and second, there are many potential goals to pursue. When potential goals are temporally distant, human decision makers cannot use an exhaustive planning strategy, rendering simpler rules of thumb more appropriate. An important question is how humans adjust the rule of thumb approach once they get closer to the goal. We addressed this question using a novel sequential two-goal task and analysed the choice data using a computational model which arbitrates between a rule of thumb and accurate planning. We found that participants decision making progressively improved as the goal came closer and that this improvement was most likely caused by participants starting to plan ahead.

animal behavior and cognition↗

bioRxiv2019

Balancing control: a Bayesian interpretation of habitual and goal-directed behavior

In everyday life, our behavior varies on a continuum from automatic and habitual to deliberate and goal-directed. Recent evidence suggests that habit formation and relearning of habits operate in a context-dependent manner: Habit formation is promoted when actions are performed in a specific context, while breaking off habits is facilitated after a context change. It is an open question how one can computationally model the brains balancing between context-specific habits and goal-directed actions. Here, we propose a hierarchical Bayesian approach for control of a partially observable Markov decision process that enables conjoint learning of habits and reward structure in a context-specific manner. In this model, habit learning corresponds to an updating of priors over policies and interacts with the learning of the outcome contingencies. Importantly, the model is solely built on probabilistic inference, which effectively provides a simple explanation of how the brain may balance contributions of habitual and goal-directed control. We illustrated the resulting behavior using agent-based simulated experiments, where we replicated several findings of devaluation, extinction, and renewal experiments, as well as the so-called two-step task which is typically used with human participants. In addition, we show how a single parameter, the habitual tendency, can explain individual differences in habit learning and the balancing between habitual and goal-directed control. Finally, we discuss the link of the proposed model to other habit learning models and implications for understanding specific phenomena in substance use disorder.

neuroscience↗