Biology subjects

van der Kooij, K.

Publications and source records attributed to van der Kooij, K..

5 recordsLinked to original sources

bioRxiv2023

The influence of movement speed on reward-based motor learning

Human movement is inevitably variable. This variability can be seen as a constraint to overcome, but it may also be a feature: being variable may result in the discovery of better movement solutions. Especially when feedback is limited to binary information on movement success or failure, variability is key for discovering which movements lead to success. Since moving faster increases variability, we aimed to answer the question whether movement speed can be harnessed to improve such reward-based motor learning. Subjects performed a stepping task in a slow and a fast session. They had to learn the gain between their step lengths and visual target distances on screen based on binary reward feedback. We successfully manipulated movement speed between sessions and participants could learn the gain in both sessions. We found no difference in learning between speed sessions, despite the fact that variability in gain increased in the fast relative to the slow session. To distinguish between different sources of variability, we estimated inevitable motor noise from the variability following successful trials. We estimated exploration as the additional variability following non-successful trials relative to following successful trials. We found no relation between variability sources and learning. In conclusion, reward-based motor learning is possible in a gain-learning task. In this task, moving faster did not lead to higher learning. Since the role of variability may differ between experimental tasks, whether movement speed can be harnessed to improve motor learning needs to be tested in other experimental tasks.

neuroscience↗

bioRxiv2023

Implicit reward-based motor learning

Binary feedback, providing information solely about task success or failure, can be sufficient to drive motor learning. While binary feedback can induce explicit adjustments in movement strategy, it remains unclear if this type of feedback also induce implicit learning. We examined this question in a center-out reaching task by gradually moving an invisible reward zone away from a visual target to a final rotation of 7.5{degrees} or 25{degrees} in a between-group design. Participants received binary feedback, indicating if the movement intersected the reward zone. By the end of the training, both groups modified their reach angle by about 95% of the rotation. We quantified implicit learning by measuring performance in a subsequent no-feedback aftereffect phase, in which participants were told to forgo any adopted movement strategies and reach directly to the visual target. The results showed a small, but robust (2-3{degrees}) aftereffect in both groups, highlighting that binary feedback elicits implicit learning. Notably, for both groups, reaches to two flanking generalization targets were biased in the same direction as the aftereffect. This pattern is at odds with the hypothesis that implicit learning is a form of use-dependent learning. Rather, the results suggest that binary feedback can be sufficient to recalibrate a sensorimotor map.

neuroscience↗

bioRxiv2023

Reinforcement-Based Processes Actively Regulate MotorExploration Along Redundant Solution Manifolds

From a babys babbling to a songbird practicing a new tune, exploration is critical to motor learning. A hallmark of exploration is the emergence of random walk behaviour along solution manifolds, where successive motor actions are not independent but rather become serially dependent. Such exploratory random walk behaviour is ubiquitous across species, neural firing, gait patterns, and reaching behaviour. Past work has suggested that exploratory random walk behaviour arises from an accumulation of movement variability and a lack of error-based corrections. Here we test a fundamentally different idea--that reinforcement-based processes regulate random walk behaviour to promote continual motor exploration to maximize success. Across three human reaching experiments we manipulated the size of both the visually displayed target and an unseen reward zone, as well as the probability of reinforcement feedback. Our empirical and modelling results parsimoniously support the notion that exploratory random walk behaviour emerges by utilizing knowledge of movement variability to update intended reach aim towards recently reinforced motor actions. This mechanism leads to active and continuous exploration of the solution manifold, currently thought by prominent theories to arise passively. The ability to continually explore muscle, joint, and task redundant solution manifolds is beneficial while acting in uncertain environments, during motor development, or when recovering from a neurological disorder to discover and learn new motor actions.

neuroscience↗

bioRxiv2019

Quantifying exploration in reward-based motor learning

Exploration in reward-based motor learning is observable in experimental data as increased variability. In order to quantify exploration, we compare three methods for estimating other sources of variability: sensorimotor noise. We use a task in which participants could receive stochastic binary reward feedback following a target-directed weight shift. Participants first performed 6 baseline blocks without feedback, and next twenty blocks alternating with and without feedback. Variability was assessed based on trial-to-trial changes in movement endpoint. We estimated sensorimotor noise by the median squared trial-to-trial change in movement endpoint for trials in which no exploration is expected. We identified three types of such trials: trials in baseline blocks, trials in the blocks without feedback, and rewarded trials in the blocks with feedback. We estimated exploration by the median squared trial-to-trial change following non-rewarded trials minus sensorimotor noise. As expected, variability was larger following non-rewarded trials than following rewarded trials. This indicates that our reward-based weight-shifting task successfully induced exploration. Most importantly, our three estimates of sensorimotor noise differed: the estimate based on rewarded trials was significantly lower than the estimates based on the two types of trials without feedback. Consequently, the estimates of exploration also differed. We conclude that the quantification of exploration depends critically on the type of trials used to estimate sensorimotor noise. We recommend the use of variability following rewarded trials.

neuroscience↗

bioRxiv2019

Practicing one thing at a time: the secret to reward-based learning?

Binary reward feedback on movement success is sufficient for learning in some simple reaching tasks, but not in some more complex ones. It is unclear what the critical conditions for learning are. Here, we ask how reward-based sensorimotor learning depends on the number of factors that are task-relevant. In a task that involves two factors, we test whether learning improves by giving feedback on each factor in a separate phase of the learning. Participants learned to perform a 3D trajectory matching task on the basis of binary reward-feedback in three phases. In the first and second phase, the reward could be based on the produced slant, the produced length or the combination of the two. In the third phase, the feedback was always based on the combination of the two factors. The results showed that reward-based learning did not depend on the number of factors that were task-relevant. Consistently, providing feedback on a single factor in the first two phases did not improve motor learning in the third phase.

neuroscience↗