Biology subjects

Haith, A. M.

Publications and source records attributed to Haith, A. M..

14 recordsLinked to original sources

bioRxiv2025

Policy-Gradient Reinforcement Learning as a General Theory of Practice-Based Motor Skill Learning

Mastering any new skill requires extensive practice, but the computational principles underlying this learning are not clearly understood. Existing theories of motor learning can explain short-term adaptation to perturbations, but offer little insight into the processes that drive gradual skill improvement through practice. Here, we propose that practice-based motor skill learning can be understood as a form of reinforcement learning (RL), specifically, policy-gradient RL, a simple, model-free method that is widely used in robotics and other continuous control settings. Here, we show that models based on policy-gradient learning rules capture key properties of human skill learning across a diverse range of learning tasks that have previously lacked any computational theory. We suggest that policy-gradient RL can provide a general theoretical framework and foundation for understanding how humans hone skills through practice.

neuroscience↗

bioRxiv2024

Switching between newly learned motor skills

Studies of cognitive flexibility suggest that switching between different tasks can entail a transient switch cost. Here, we asked whether analogous switch costs exist in the context of switching between different motor skills. We tested whether participants could switch between a newly learned skill associated with a novel visuomotor mapping, and an existing skill associated with an intuitive mapping. Participants showed increased errors in trials immediately following a switch between mappings. These errors were attributable to persisting with the pre-switch policy, rather than imperfect implementation or retrieval of the post-switch policy. A subset of our participants further learned a second new skill. Switching between these two novel skills was initially very challenging, but improved with further training. Our findings suggest that switching between newly learned motor skills can be challenging, and that errors in the context of switching between skills are primarily attributable to perseveration with the wrong control policy.

neuroscience↗

bioRxiv2023

Continuous motor skills as flexible control policies: a video game study

Many motor skills consist of continuous sequential actions, such as a tennis serve. It is currently unclear how these surprisingly understudied behaviors are learned, with the leading hypothesis being that sequences of single actions become "chunked" into larger single executable units. Under this hypothesis, continuous sequential actions should become more task-specific and less generalizable with practice. To test this, we developed a video game that requires participants to hold a tablet with both hands and steer a virtual car (the "ant car") along a curving track. We tested participants ability to generalize their skill to a probe track that required a different sequence of turns. Across days of practice, task success increased, and movement variability decreased. On the probe track, movement quality at the level of kinematics fully generalized but performance at the level of task success showed a consistent decrement. To address this apparent paradox, we empirically derived the control policy participants used at their maximal skill level on the training track. Notably, this policy was fully transferred to the probe track, but there were more instances of momentary deviations from it (lapses), which explains the worse performance despite equivalent skill. We conclude that continuous motor skills are acquired through learning of a flexible control policy that maps states onto actions and not through chunking or automatizing of a specific sequence of actions.

neuroscience↗

bioRxiv2022

Action inhibition revisited: Stopping is not faster than going

Much work has examined the process of canceling or inhibiting an impending action. However, the converse process of initiating a movement has, comparatively, been much less studied. Action initiation and action inhibition are generally considered to be independent and qualitatively distinct processes, while the exact relationship between these processes remains unclear. One respect in which action initiation and action inhibition are often thought to differ is in their speed; action inhibition is typically considered to be faster than action initiation, which allows for impending actions to be inhibited before they are initiated. This apparent contrast is, however, largely observed in tasks in which there is much greater urgency to inhibit an action that there is to initiate an action. This asymmetry in the urgency between action initiation and action inhibition, as well as other asymmetries in how action initiation and inhibition are cued, make it impossible to compare their relative time courses. Here, we demonstrate that, when action initiation and action inhibition are measured under conditions that are matched as closely as possible, their speed is the same. In light of this, we suggest that action initiation and action inhibition may not necessarily be qualitatively distinct processes, but may instead reflect two opposing states of a single process supporting a decision about whether to act or not. This perspective carries significant implications for computational models and presumed neural mechanisms of action control.

neuroscience↗

bioRxiv2022

Evidence for a common mechanism supporting invigoration of action selection and action execution

The speed, or vigor, of our movements can vary depending on circumstances. For instance, the promise of a reward leads to faster movements. Reward also leads us to move with a lower reaction time, suggesting that the process of action selection can also be invigorated by reward. It has been proposed that invigoration of action selection and of action execution might occur through a common mechanism, and thus these aspects of behavior might be coupled. To test this hypothesis, we asked participants to make reaching movements to "shoot" through a target at varying speeds to assess whether moving more quickly was also associated with more rapid action selection. We found that, when participants were required to move with a lower velocity, the speed of their action selection was also significantly slowed. This finding was recapitulated in a further dataset in which participants determined their own movement speed, but had to move slowly in order to stop their movement inside the target. By re-analyzing a previous dataset, we also found evidence for the converse relationship between action execution and action selection: when pressured to select actions more rapidly, people also executed movements with higher velocity. Our results establish that invigoration of action selection and action execution vary in tandem with one another, supporting the hypothesis of a common underlying mechanism. Significance statementWe show that voluntary increases in the vigor of action execution lead action selection to also occur more rapidly. Conversely, hastening action selection by imposing a deadline to act also leads to increases in movement speed. These findings provide evidence that these two distinct aspects of behavior are modulated by a common underlying mechanism.

neuroscience↗

bioRxiv2022

Control becomes habitual early on when learning a novel motor skill

When people perform the same task repeatedly, their behavior becomes habitual, or inflexible to changes in the goals or structure of a task. While habits have been hypothesized to be a key aspect of motor skill acquisition, there has been little empirical work investigating the relationship between skills and habits. To better understand this relationship, we examined whether and when peoples behavior would become habitual as they learned a challenging new motor skill. After up to ten days of practice, we altered the structure of the task to assess whether participants would flexibly adjust their behavior or habitually persist in performing the task the way they originally learned. We found that participants behavior became habitual early in practice--after only two days--at which point they were still relatively unskilled. These data demonstrate that motor skills become habitual after relatively little training, but can nevertheless further improve with practice.

neuroscience↗

bioRxiv2021

Proprioceptive Re-alignment drives Implicit Sensorimotor Adaptation

Multiple learning processes contribute to successful goal-directed actions in the face of changing physiological states, biomechanical constraints, and environmental contexts. Amongst these processes, implicit sensorimotor adaptation is of primary importance, ensuring that movements remain well-calibrated and accurate. A large body of work on reaching movements has emphasized how adaptation centers on an iterative process designed to minimize visual errors. The role of proprioception has been largely neglected, thought to play a passive role in which proprioception is affected by the visual error but does not directly contribute to adaptation. Here we present an alternative to this visuo-centric framework, arguing that that implicit adaptation can be understood as minimizing a proprioceptive error, the distance between the perceived hand position and its intended goal. We use this proprioceptive re-alignment model (PReMo) to re-examine many phenomena that have previously been interpreted in terms of learning from visual errors, as well as offer novel accounts for unexplained phenomena. We discuss potential challenges for this new perspective on implicit adaptation and outline a set of predictions for future experimentation.

neuroscience↗

bioRxiv2021

De novo motor learning of a bimanual control task over multiple days of practice

Although much research on motor learning has focused on how we adapt our movements to maintain performance in the face of imposed perturbations, in many cases we must learn new skills from scratch, or de novo. In comparison to adaptation, relatively little is known about de novo learning. In part, this is because learning a new skill can involve many challenges, including learning to recognize new patterns of sensory input and generate new patterns of motor output. However, even with familiar sensory cues and well-practiced movements, the problem of quickly selecting the appropriate actions in response to the current state is challenging. Here, we devised a bimanual hand-to-cursor mapping which isolates this control problem. We find that participants initially struggled to control the cursor under this bimanual mapping, despite explicit knowledge of the mapping. Performance improved steadily over multiple days of practice, however. Participants exhibited no aftereffects when reverting to a veridical cursor, confirming that participants learned the new task de novo, rather than through adaptation. Corrective responses to mid-movement perturbations of the target were initially weak, but with practice, participants gradually became able to respond rapidly and robustly to these perturbations. After four days of practice, participants behavior under the bimanual mapping almost matched performance with a veridically mapped cursor. However, there remained a small but persistent difference in performance level. Our findings illustrate the dynamics and limitations of learning a novel controller and introduce a promising paradigm for tractably investigating this aspect of motor skill learning.

neuroscience↗

bioRxiv2021

Age-related increases in reaction time result from slower preparation, not delayed initiation

Recent work indicates that healthy younger adults can prepare accurate responses faster than their voluntary reaction times indicate, leaving a seemingly unnecessary delay of 80-100ms before responding. Here we examined how the preparation of movements, initiation of movements, and the delay between them are affected by ageing. Participants made planar reaching movements in two conditions. The Free Reaction Time condition assessed the voluntary reaction times with which participants responded to the appearance of a stimulus. The Forced Reaction Time condition assessed the minimum time actually needed to prepare accurate movements by controlling the time allowed for movement preparation. The time taken to both initiate movements in the Free Reaction Time and to prepare movements in the Forced Response condition increased with age. Notably, the time required to prepare accurate movements was significantly shorter than participants self-selected initiation times; however, the delay between movement preparation and initiation remained consistent across the lifespan (~90ms). These results indicate that the slower reaction times of healthy older adults are not due to an increased hesitancy to respond, but can instead be attributed to changes in their ability to process stimuli and prepare movements accordingly, consistent with age-related changes in brain structure and function.

neuroscience↗

bioRxiv2020

Competition between parallel sensorimotor learning systems

Sensorimotor adaptation benefits from learning in two parallel systems: one that has access to explicit knowledge, and another that relies on implicit, unconscious correction. However, it is unclear how these systems interact: does enhancing one systems contributions, for example through instruction, impair the other, or do they learn independently? Here we illustrate that certain contexts can lead to competition between implicit and explicit learning. In some cases, each system is responsive to a task-related visual error. This shared error appears to create competition between these systems, such that when the explicit system increases its response, errors are siphoned away from the implicit system, thus reducing its learning. This model suggests that explicit strategy can mask changes in implicit error sensitivity related to savings and interference. Other contexts suggest that the implicit system can respond to multiple error sources. When these error sources conflict, a second type of competition occurs. Thus, the data show that during sensorimotor adaptation, behavior is shaped by competition between parallel learning systems.

neuroscience↗

bioRxiv2020

Implicit adaptation is driven by direct policy updates rather than forward-model-based learning

The human motor system can rapidly adapt its motor output in response to errors. The prevailing theory of this process posits that the motor system adapts an internal forward model that predicts the consequences of outgoing motor commands, and that this forward model is then used to guide selection of motor output. However, although there is clear evidence for the existence of adaptive forward models to help track the state of the body, there is no real evidence that such models influence the selection of motor output. A possible alternative to the forward-model-based theory of adaptation is that motor output could be directly adjusted by movement errors ("direct policy learning"), in parallel with but independent of any updates to a predictive forward model. Here, we show evidence for this latter theory based on the properties of implicit adaptation under mirror-reversed visual feedback. We show that implicit adaptation still occurs under this extreme perturbation but acts in an inappropriate direction, following a pattern consistent with direct policy learning but not forward-model-based learning. We suggest that the forward-model-based theory of adaptation needs to be re-examined and that direct policy learning is a more plausible mechanism of implicit adaptation.

neuroscience↗

bioRxiv2020

De novo learning versus adaptation of continuous control in a manual tracking task

How do people learn to perform tasks that require continuous adjustments of motor output, like riding a bicycle? People rely heavily on cognitive strategies when learning discrete movement tasks, but such time-consuming strategies are infeasible in continuous control tasks that demand rapid responses to ongoing sensory feedback. To understand how people can learn to perform such tasks without the benefit of cognitive strategies, we imposed a rotation/mirror reversal of visual feedback while participants performed a continuous tracking task. We analyzed behavior using a system identification approach which revealed two qualitatively different components of learning: adaptation of a baseline controller and formation of a new, task-specific continuous controller. These components exhibited different signatures in the frequency domain and were differentially engaged under the rotation/mirror reversal. Our results demonstrate that people can rapidly build a new continuous controller de novo and can simultaneously deploy this process with adaptation of an existing controller.

neuroscience↗

bioRxiv2017

Practice induces a qualitative change in the memory representation for visuomotor learning

Adaptation of our movements to changes in the environment is known to be supported by multiple learning processes which act in parallel. An implicit process recalibrates motor output to maintain alignment between intended and observed movement outcomes (\"implicit recalibration\"). In parallel, an explicit learning process drives more strategic adjustments of behavior, often by deliberately aiming movements away from an intended target (\"deliberate re-aiming\"). It has long been established that people form a memory for prior experience adapting to a perturbation through the fact that they become able to more rapidly adapt to familiar perturbations (a phenomenon known as \"savings\"). Repeated exposures to the same perturbation can further strengthen savings. It remains unclear, however, which underlying learning process is responsible for this practice-related improvement in savings. We measured the relative contributions of implicit recalibration and deliberate re-aiming to adaptation during multiple exposures to an alternating sequence of perturbations over two days. We found that the implicit recalibration followed an invariant learning curve despite prolonged practice, and thus exhibited no memory of prior experience. Instead, practice led to a qualitative change in re-aiming which, in addition to supporting savings, became able to be expressed rapidly and automatically. This qualitative change appeared to enable participants to form memories for two opposing perturbations, overcoming interference effects that typically prohibit savings when learning multiple, opposing perturbations. Our results are consistent with longstanding theories that frame skill learning as a transition from deliberate to automatic selection of actions.

neuroscience↗

bioRxiv2017

Skill Acquisition and Habit Formation as Distinct Effects of Practice

Converging evidence indicates that separate goal-directed and habitual systems compete to control behavior1. However, it has proven difficult to reliably induce habitual behavior in human participants2-4. We reasoned that habits may be present in the form of habitually prepared responses, but are overridden by goal-directed processes, preventing their overt expression. Here we show that latent habits can be unmasked by limiting the time participants have to respond to a stimulus. Participants trained for 4 days on a visuomotor association task. By continuously varying the time allowed to prepare responses, we found that the probability of expressing a learned habit followed a stereotyped time course, peaking 300-600ms after stimulus presentation. This time course was captured by a computational model of response preparation in which habitual responses are automatically prepared at short latency, but are replaced by goal-directed responses at longer latency. A more extensive period of practice (20 days) led to increased habit expression by reducing the average time of movement initiation. These findings refine our understanding of habits, and show that practice can influence habitual behavior in distinct ways: by promoting habit formation, and by modulating the likelihood of habit expression.

neuroscience↗