Biology subjects

Goschke, T.

Publications and source records attributed to Goschke, T..

5 recordsLinked to original sources

bioRxiv2019

Individual differences in dopamine function underlying the balance between model-based and model-free control

Reinforcement learning involves a balance between model-free (MF) and model-based (MB) systems. Recent studies suggest that individuals with either pharmacologically enhanced levels of dopamine (DA) or higher baseline levels of DA exhibit more MB control. However, it remains unknown whether such pharmacological effects depend on baseline DA. Here, we investigated whether effects of L-DOPA on the balance of MB/MF control depend on ventral striatal baseline DA. Sixty participants had two functional magnetic resonance imaging (fMRI) scans while performing a two-stage sequential decision-making task under 150 mg L-DOPA or placebo (counterbalanced), followed by a 4-hour 18F-DOPA positron emission tomography (PET) scan (on a separate occasion). We found an interaction between baseline DA levels and L-DOPA induced changes in MB control. Individuals with higher baseline DA levels showed a greater L-DOPA induced enhancement in MB control. Surprisingly, we found a corresponding drug-by-baseline DA interaction on MF, but not MB learning signals in the ventromedial prefrontal cortex. We did not find a significant interaction between baseline DA levels and L-DOPA effects on MF control or MB/MF balance. In sum, our findings point to a baseline dependency of L-DOPA effects on differential aspects of MB and MF control. Individual differences in DA washout may be an important moderator of L-DOPA effects. Overall, our findings complement the general notion where higher DA levels is related to a greater reliance on MB control. Although the relationship between phasic DA firing and MF learning is conventionally assumed in the animal literature, the relationship between DA and MF control is not as straightforward and requires further clarification.

neuroscience↗

bioRxiv2019

Meta-control of the exploration-exploitation dilemma emerges from probabilistic inference over a hierarchy of time scales

Cognitive control is typically understood as a set of mechanisms which enable humans to reach goals that require integrating the consequences of actions over longer time scales. Importantly, using routine beheavior or making choices beneficial only at a short time scales would prevent one from attaining these goals. During the past two decades, researchers have proposed various computational cognitive models that successfully account for behaviour related to cognitive control in a wide range of laboratory tasks. As humans operate in a dynamic and uncertain environment, making elaborate plans and integrating experience over multiple time scales is computationally expensive, the specific question of how uncertain consequences at different time scales are integrated into adaptive decisions remains poorly understood. Here, we propose that precisely the problem of integrating experience and forming elaborate plans over multiple time scales is a key component for better understanding how human agents solve cognitive control dilemmas such as the exploration-exploitation dilemma. In support of this conjecture, we present a computational model of probabilistic inference over hidden states and actions, which are represented as a hierarchy of time scales. Simulations of goal-reaching agents instantiating the model in an uncertain and dynamic task environment show how the exploration-exploitation dilemma may be solved by inferring meta-control states which adapt behaviour to changing contexts.

neuroscience↗

bioRxiv2019

Modeling dynamic allocation of effort in a sequential task using discounting models

Most rewards in our lives require effort to obtain them. It is known that effort is seen by humans as carrying an intrinsic disutility which devalues the obtainable reward. Established models for effort discounting account for this by using participant-specific discounting parameters inferred from experiments. These parameters offer only a static glance into the bigger picture of effort exertion. The mechanism underlying the dynamic changes in a participants willingness to exert effort is still unclear and an active topic of research. Here, we modeled dynamic effort exertion as a consequence of effort- and probability-discounting mechanisms during goal reaching, sequential behavior. To do this, we developed a novel sequential decision-making task in which participants make binary choices to reach a minimum number of points. Importantly, the time points and circumstances of effort allocation are decided by participants according to their own preferences and not imposed directly by the task. Using the computational model to analyze participants choices, we show that the dynamics of effort exertion arise from a combination of changing task needs and forward planning. In other words, the interplay between a participants inferred discounting parameters is sufficient to explain the dynamic allocation of effort during goal reaching. Using formal model comparison, we also infer the forward-planning strategy used by participants. The model allows us to characterize a participants effort exertion in terms of only a few parameters. Moreover, the model can be adapted to a number of tasks used in establishing the neural underpinnings of forward-planning behavior and meta-control, allowing for the characterization of behavior in terms of model parameters.

neuroscience↗

bioRxiv2017

Switch Independent Task Representations In Frontal And Parietal Cortex

Alternating between two tasks is effortful and impairs performance. Previous functional magnetic resonance imaging (fMRI) studies have found increased activity in fronto-parietal cortex when task switching is required. One possibility is that the additional control demands for switch trials are met by strengthening task representations in the human brain. Alternatively, on switch trials the residual representation of the previous task might impede the buildup of a neural task representation. This would predict weaker task representations on switch trials, thus also explaining the performance costs. To test this, participants were cued to perform one of two similar tasks, with the task being repeated or switched between successive trials. MVPA was used to test which regions encode the tasks and whether this encoding differs between switch and repeat trials. As expected, we found information about task representations in frontal and parietal cortex, but there was no difference in the decoding accuracy of task-related information between switch and repeat trials. Using cross-classification we found that the fronto-parietal cortex encodes tasks using a similar spatial pattern in switch and repeat trials. Thus, task representations in frontal and parietal cortex are largely switch-independent. We found no evidence that neural information about task representations in these regions can explain behavioral costs usually associated with task switching.\n\nSignificance statementAlternating between two tasks is effortful and slows down performance. One possible explanation is that the representations in the human brain need time to build up and are thus weaker on switch trials, explaining performance costs. Alternatively, task representations might even be enhanced in order to overcome the previous task. Here we used a combination of fMRI and a brain classifier to test whether the additional control demands under switching conditions lead to an increased or decreased strength of task representations in fronto-parietal brain regions. We found that task representations are not significantly modulated by switching processes. Thus, task representations in the human brain cannot account for the performance costs associated with alternating between tasks.

neuroscience↗

bioRxiv2016

L-DOPA reduces model-free control of behavior by attenuating the transfer of value to action

Dopamine is a key neurotransmitter in reinforcement learning and action control. Recent findings suggest that these components are inherently entangled. Here, we tested if increases in dopamine tone by administration of L-DOPA upregulate deliberative \"model-based\" control of behavior or reflexive \"model-free\" control as predicted by dual-control reinforcement-learning models. Alternatively, L-DOPA may impair learning as suggested by \"value\" or \"thrift\" theories of dopamine. To this end, we employed a two-stage Markov decision-task to investigate the effect of L-DOPA (randomized cross-over) on behavioral control while brain activation was measured using fMRI. L-DOPA led to attenuated model-free control of behavior as indicated by the reduced impact of reward on choice and increased stochasticity of model-free choices. Correspondingly, in the brain, L-DOPA decreased the effect of reward while prediction-error signals were unaffected. Taken together, our results suggest that L-DOPA reduces model-free control of behavior by attenuating the transfer of value to action.

neuroscience↗