Continuous monitoring of cost-to-go for flexible reaching control and online decisions

Abstract

Humans consider the parameters linked to movement goal during reaching to adjust their control strategy online. Indeed, rapid changes in target structure or disturbances interfering with their initial plan elicit rapid changes in behavior. Here, we hypothesize that these changes could result from the continuous use of a decision variable combining motor and cognitive components. We combine an optimal feedback controller with a real-time monitoring of the expected cost-to-go, which considers target- and movement-related costs, in a common theoretical framework. This model reproduces human behaviors in presence of changes in the target structure occurring during movement and of online decisions to flexibly change target following external perturbations. It also predicts that the time taken to decide to select a novel goal after a perturbation depends on the amplitude of the disturbance and on the rewards of the different options, which is a direct result of the continuous monitoring of the cost-to-go. We show that this result was present in our previously collected dataset. Together our developments point towards a continuous monitoring of the cost-to-go during reaching to update control online and make efficient decisions about movement goal. Author summaryThe way humans perform reaching movements is compatible with models considering that they result from the minimization of a task-related cost function. However, these models typically assume a cost function that does not change within movement, which is incompatible with experimental findings highlighting humans ability to adjust reaching control online and change target flexibly. We hypothesized that this later ability relied on the cost-to-go, which integrates task- and body-related information, being evaluated continuously during movement. We show that this model can optimally select and adjust control during movement in a way that reproduces human behavior in a set of tasks involving change in cost function and change in goal target. Our model predicted that decision-time to change target must be postponed when limb displacements and alternative rewards are smaller, which was borne out in our previous experimental dataset. To conclude, our model explains dynamic updates in reach control and suggests the cost-to-go as decision variable linking decision-making and motor control.