Hayhoe, M. M. | Search bioRxiv

bioRxiv2020

Retinal optic flow during natural locomotion

We examine the structure of visual motion on the retina during natural locomotion in real world environments. Natural locomotion generates a rhythmic translation and rotation profile of the head in space, which means that visually specified heading varies throughout the gait cycle. This presents a challenge if optic flow is to be used to control heading towards a goal. The complex, phasic head movements that occur through the gait cycle create a highly unstable pattern of flow relative to the head. In contrast, vestibular-ocular-reflex mediated fixation simplifies patterns of optic flow on the retinae, resulting in regular features that may be valuable for the control of locomotion. In particular, the sign and magnitude of foveal curl in retinal flow fields specifies the bodys trajectory relative to the fixation point. In addition, the peak in the divergence of the retinal flow field specifies the walkers instantaneous overground velocity/momentum vector in retinotopic coordinates. Assuming that walkers can determine the body position relative to fixation, this time-varying retinotopic cue for the bodys momentum could provide a visual control signal for foot placement over complex terrain. In contrast, the temporal variation of heading is large enough to be problematic for use in steering towards a goal. Consideration of optic flow in the context of natural locomotion therefore suggests a re-evaluation of the role of optic flow in the visual control of locomotion.

neuroscience↗

bioRxiv2018

Modeling Sensory-Motor Decisions in Natural Behavior

Although a standard reinforcement learning model can capture many aspects of reward-seeking behaviors, it may not be practical for modeling human natural behaviors because of the richness of dynamic environments and limitations in cognitive resources. We propose a modular reinforcement learning model that addresses these factors. Based on this model, a modular inverse reinforcement learning algorithm is developed to estimate both the rewards and discount factors from human behavioral data, which allows predictions of human navigation behaviors in virtual reality with high accuracy across different subjects and with different tasks. Complex human navigation trajectories in novel environments can be reproduced by an artificial agent that is based on the modular model. This model provides a strategy for estimating the subjective value of actions and how they influence sensory-motor decisions in natural behavior.\n\nAuthor summaryIt is generally agreed that human actions can be formalized within the framework of statistical decision theory, which specifies a cost function for actions choices, and that the intrinsic value of actions is controlled by the brains dopaminergic reward machinery. Given behavioral data, the underlying subjective reward value for an action can be estimated through a machine learning technique called inverse reinforcement learning. Hence it is an attractive method for studying human reward-seeking behaviors. Standard reinforcement learning methods were developed for artificial intelligence agents, and incur too much computation to be a viable model for real-time human decision making. We propose an approach called modular reinforcement learning that decomposes a complex task into independent decision modules. This model includes a frequently overlooked variable called the discount factor, which controls the degree of impulsiveness in seeking future reward. We develop an algorithm called modular inverse reinforcement learning that estimates both the reward and the discount factor. We show that modular reinforcement learning may be a useful model for natural navigation behaviors. The estimated rewards and discount factors explain human walking direction decisions in a virtual-reality environment, and can be used to train an artificial agent that can accurately reproduce human navigation trajectories.

neuroscience↗