Biology subjects

Howard, M. W.

Publications and source records attributed to Howard, M. W..

4 recordsLinked to original sources

bioRxiv2018

Predicting the future with multi-scale successor representations

The successor representation (SR) is a candidate principle for generalization in reinforcement learning, computational accounts of memory, and the structure of neural representations in the hippocampus. Given a sequence of states, the SR learns a predictive representation for every given state that encodes how often, on average, each upcoming state is expected to be visited, even if it is multiple steps ahead. A discount or scale parameter determines how many steps into the future SR's generalizations reach, enabling rapid value computation, subgoal discovery, and flexible decision-making in large trees. However, SR with a single scale could discard information for predicting both the sequential order of and the distance between states, which are common problems in navigation for animals and artificial agents. Here we propose a solution: an ensemble of SRs with multiple scales. We show that the derivative of multi-scale SR can reconstruct both the sequence of expected future states and estimate distance to goal. This derivative can be computed linearly: we show that a multi-scale SR ensemble is the Laplace transform of future states, and the inverse of this Laplace transform is a biologically plausible linear estimation of the derivative. Multi-scale SR and its derivative could lead to a common principle for how the medial temporal lobe supports both map-based and vector-based navigation.

neuroscience↗

bioRxiv2017

Scanning along a compressed timeline of the future

Several authors have suggested a deep symmetry between the psychological processes that underlie our ability to remember the past and make predictions about the future. The judgment of recency (JOR) task measures temporal order judgments for the past by presenting pairs of probe stimuli; participants choose the probe that was presented more recently. We performed a short-term relative JOR task and introduced a novel judgment of imminence (JOI) task to study temporal order judgments for the future. In the JOR task, participants were presented with a sequence of stimuli and asked to choose which of two probe stimuli was presented closer to the present. In the JOI task, participants were trained on a probabilistic sequence. After training, the sequence was interrupted with probe stimuli. Participants were asked to choose which of two probe stimuli was expected to be presented closer to the present. Replicating prior work on JOR, we found that RT results supported a backward self-terminating search model operating on a temporally-organized representation of the past. We also showed that RT distributions are consistent with this model and that the temporally-organized representation is compressed. Critically, results for the JOI task probing expectations of the future were mirror-symmetric to results from memory, suggesting a forward self-terminating search model operating on a temporally-organized representation of the future.

animal behavior and cognition↗

bioRxiv2017

Recency Order Judgments In Short Term Memory: Replication And Extension Of Hacker (1980)

The classic finding from short-term relative JOR tasks is that correct response time (RT) depends on the lag to the more recent item but not to the less recent item (Hacker, 1980). For decades, researchers have argued that this finding is consistent with a self-terminating backward scanning model (Muter, 1979; Hacker, 1980; Hockley, 1984; McElree & Dosher, 1993). This finding has taken on new importance in light of recent proposal that many forms of memory depend on a compressed representation of the past (Howard, Shankar, Aue, & Criss, 2015). This paper replicates and extends the results of the classic papers. A Bayesian t-test showed substantial evidence for the null effect of lag to the less recent item on correct RT. In addition, this paper reports that correct RT is a sub-linear function of lag to the more recent probe and replicates the classic finding that error RT depends on lag to the less recent probe. These findings place new constraints on models of short-term memory scanning.

animal behavior and cognition↗

bioRxiv2017

Compressed timeline of recent experience in monkey lPFC

Cognitive theories suggest that working memory maintains not only the identity of recently-presented stimuli but also a sense of the elapsed time since the stimuli presentation. Previous studies of the neural underpinnings of working memory have focused on sustained firing, which can account for maintenance of the stimulus identity, but not for representation of the elapsed time. We analyzed single-unit recordings from the macaque lateral prefrontal cortex (lPFC) during performance of a delayed-match-to-category task. The sample stimulus triggered a consistent sequence of neurons, each neuron in the sequence firing during a circumscribed period of time. The sequences initiated by different sample stimuli were distinct but overlapping, with the degree of overlap reflecting the visual similarity of the stimuli that caused the sequences. These sequences of neurons encoded both stimulus identity and the elapsed time. The temporal code became less precise as the sample stimulus receded into the past. These findings suggest that working memory is maintained as a compressed timeline, consistent with longstanding cognitive theories of human memory.

neuroscience↗