Biology subjects

Keramati, M.

Publications and source records attributed to Keramati, M..

3 recordsLinked to original sources

bioRxiv2017

Flexibility to contingency changes distinguishes habitual and goal-directed strategies in humans

Decision-making in the real world presents the challenge of requiring flexible yet prompt behavior, a balance that has been characterized in terms of a trade-off between a slower, prospective goal-directed model-based (MB) strategy and a fast, retrospective habitual model-free (MF) strategy. Theory predicts that flexibility to changes in both reward values and transition contingencies can determine the relative influence of the two systems in reinforcement learning, but few studies have manipulated the latter. Therefore, we developed a novel two-level contingency change task in which transition contingencies between states change every few trials; MB and MF control predict different responses following these contingency changes, allowing their relative influence to be inferred. Additionally, we manipulated the rate of contingency changes in order to determine whether contingency change volatility would play a role in shifting subjects between a MB and MF strategy. We found that human subjects employed a hybrid MB/MF strategy on the task, corroborating the parallel contribution of MB and MF systems in reinforcement learning. Further, subjects did not remain at one level of MB/MF behavior but rather displayed a shift towards more MB behavior over the first two blocks that was not attributable to the rate of contingency changes but rather to the extent of training. We demonstrate that flexibility to contingency changes can distinguish MB and MF strategies, with human subjects utilizing a hybrid strategy that shifts towards more MB behavior over blocks, consequently corresponding to a higher payoff.\n\nAuthor SummaryTo make good decisions, we must learn to associate actions with their true outcomes. Flexibility to changes in action/outcome relationships, therefore, is essential for optimal decision-making. For example, actions can lead to outcomes that change in value - one day, your favorite food is poorly made and thus less pleasant. Alternatively, changes can occur in terms of contingencies - ordering a dish of one kind and instead receiving another. How we respond to such changes is indicative of our decision-making strategy; habitual learners will continue to choose their favorite food even if the quality has gone down, whereas goal-directed learners will soon learn it is better to choose another dish. A popular paradigm probes the effect of value changes on decision making, but the effect of contingency changes is still unexplored. Therefore, we developed a novel task to study the latter. We find that humans used a mixed habitual/goal-directed strategy in which they became more goal-directed over the course of the task, and also earned more rewards with increasing goal-directed behavior. This shows that flexibility to contingency changes is adaptive for learning from rewards, and indicates that flexibility to contingency changes can reveal which decision-making strategy is used.

animal behavior and cognition↗

bioRxiv2017

Stochastic satisficing account of choice and confidence in uncertain value-based decisions

Every day we make choices under uncertainty; choosing what route to work or which queue in a supermarket to take, for example. It is unclear how outcome variance, e.g. uncertainty about waiting time in a queue, affects decisions and confidence when outcome is stochastic and continuous. How does one evaluate and choose between an option with unreliable but high expected reward, and an option with more certain but lower expected reward? Here we used an experimental design where two choices payoffs took continuous values, to examine the effect of outcome variance on decision and confidence. We found that our participants probability of choosing the good (high expected reward) option decreased when the good or the bad options payoffs were more variable. Their confidence ratings were affected by outcome variability, but only when choosing the good option. Unlike perceptual detection tasks, confidence ratings correlated only weakly with decisions time, but correlated with the consistency of trial-by-trial choices. Inspired by the satisficing heuristic, we propose a \"stochastic satisficing\" (SSAT) model for evaluating options with continuous uncertain outcomes. In this model, options are evaluated by their probability of exceeding an acceptability threshold, and confidence reports scale with the chosen options thus-defined satisficing probability. Participants decisions were best explained by an expected reward model, while the SSAT model provided the best prediction of decision confidence. We further tested and verified the predictions of this model in a second experiment. Our model and experimental results generalize the models of metacognition from perceptual detection tasks to continuous-value based decisions. Finally, we discuss how the stochastic satisficing account of decision confidence serves psychological and social purposes associated with the evaluation, communication and justification of decision-making.\n\nAuthor SummaryEvery day we make several choices under uncertainty, like choosing a queue in a supermarket. However, the computational mechanisms underlying such decisions remain unknown. For example, how does one choose between an option with unreliable high expected reward, like the volatile express queue, and an option with more certain but lower expected reward in the standard queue? Inspired by bounded rationality and the notion of satisficing, i.e. settling for a good enough option, we propose that such decisions are made by comparing the likelihood of different actions to surpass an acceptability threshold. When facing uncertain decisions, our participants confidence ratings were not consistent with the expected outcomes rewards, but instead followed the satisficing heuristic proposed here. Using an acceptability threshold may be especially useful when evaluating and justifying decisions under uncertainty.

neuroscience↗

bioRxiv2017

The idiosyncratic nature of confidence

Confidence is the feeling of knowing that accompanies decision making and guides processes such as learning, error detection, and inter-personal communication. Bayesian theory proposes that confidence is a function of the probability that a decision is correct given the evidence. Empirical research has shown, however, that humans tend to report confidence in very different ways. This idiosyncratic behaviour suggests that different individuals may perform different computations to estimate confidence from uncertain evidence. We tested this hypothesis by collecting confidence reports from healthy adults making decisions under either visual or numerical uncertainty. We found that for most individuals, confidence did indeed reflect the perceived probability of being correct. However, in approximately half of them, confidence also reflected a different probabilistic quantity: the observed Fisher information. We isolated the influence of each of these two quantities on confidence, and found that this decomposition is stable across weeks, and consistent across tasks involving uncertainty in both perceptual and cognitive domains. Our findings provide, for the first time, a mechanistic interpretation of individual differences in the human sense of confidence.

neuroscience↗