Biology subjects

Pitkow, X.

Publications and source records attributed to Pitkow, X..

29 records · Page 2Linked to original sources

bioRxiv2021

Petascale neural circuit reconstruction: automated methods

3D electron microscopy (EM) has been successful at mapping invertebrate nervous systems, but the approach has been limited to small chunks of mammalian brains. To scale up to larger volumes, we have built a computational pipeline for processing petascale image datasets acquired by serial section EM, a popular form of 3D EM. The pipeline employs convolutional nets to compute the nonsmooth transformations required to align images of serial sections containing numerous cracks and folds, detect neuronal boundaries, label voxels as axon, dendrite, soma, and other semantic categories, and detect synapses and assign them to presynaptic and postsynaptic segments. The output of neuronal boundary detection is segmented by mean affinity agglomeration with semantic and size constraints. Pipeline operations are implemented by leveraging distributed and cloud computing. Intermediate results of the pipeline are held in cloud storage, and can be effortlessly viewed as images, which aids debugging. We applied the pipeline to create an automated reconstruction of an EM image volume spanning four visual cortical areas of a mouse brain. Code for the pipeline is publicly available, as is the reconstructed volume.

neuroscience↗

bioRxiv2021

Sensory evidence accumulation using optic flow in a naturalistic navigation task

Sensory evidence accumulation is considered a hallmark of decision-making in noisy environments. Integration of sensory inputs has been traditionally studied using passive stimuli, segregating perception from action. Lessons learned from this approach, however, may not generalize to ethological behaviors like navigation, where there is an active interplay between perception and action. We designed a sensory-based sequential decision task in virtual reality in which humans and monkeys navigated to a memorized location by integrating optic flow generated by their own joystick movements. A major challenge in such closed-loop tasks is that subjects actions will determine future sensory input, causing ambiguity about whether they rely on sensory input rather than expectations based solely on a learned model of the dynamics. To test whether subjects performed sensory integration, we used three independent experimental manipulations: unpredictable optic flow perturbations, which pushed subjects off their trajectory; gain manipulation of the joystick controller, which changed the consequences of actions; and manipulation of the optic flow density, which changed the reliability of sensory evidence. Our results suggest that both macaques and humans relied heavily on optic flow, thereby demonstrating a critical role for sensory evidence accumulation during naturalistic action-perception closed-loop tasks.

neuroscience↗

bioRxiv2021

What The Brain Does As We Speak

AO_SCPLOWBSTRACTC_SCPLOWLanguage is a defining human behavior and is dependent on networks interactions amongst broadly distributed neuronal substrates. Transient dynamics between language regions that underpin speech production have long been postulated, yet have proven challenging to evaluate empirically. We used direct intracranial recordings during single word production to create a finely resolved spatiotemporal atlas (134 patients, 25810 electrodes, 40278 words) of the entire language-dominant cortex and used this to derive single-trial state-space sequences of network motifs. We derived 5 discrete neural states during the production of each word, distinguished by unique patterns of distributed cortical interaction. This interactive model was significantly better than a model of the same design but lacking interactions between regions in explaining observed activity. Our results eschew strict functional attribution to localized cortical populations, supporting instead the idea that cognitive processes are better explained by distributed metastable network states.

neuroscience↗

bioRxiv2020

Influence of sensory modality and control dynamics on human path integration

Path integration is a sensorimotor computation that can be used to infer latent dynamical states by integrating self-motion cues. We studied the influence of sensory observation (visual/vestibular) and latent control dynamics (velocity/acceleration) on human path integration using a novel motion-cueing algorithm. Sensory modality and control dynamics were both varied randomly across trials, as participants controlled a joystick to steer to a memorized target location in virtual reality. Visual and vestibular steering cues allowed comparable accuracies only when participants controlled their acceleration, suggesting that vestibular signals, on their own, fail to support accurate path integration in the absence of sustained acceleration. Nevertheless, performance in all conditions reflected a failure to fully adapt to changes in the underlying control dynamics, a result that was well explained by a bias in the dynamics estimation. This work demonstrates how an incorrect internal model of control dynamics affects navigation in volatile environments in spite of continuous sensory feedback.

neuroscience↗

bioRxiv2019

Population coding of strategic variables during foraging in freely-moving macaques

Until now it has been difficult to examine the neural bases of foraging in naturalistic environments because previous approaches have relied on restrained animals performing trial-based foraging tasks. Here, we allowed unrestrained monkeys to freely interact with concurrent reward options while we wirelessly recorded population activity in dorsolateral prefrontal cortex (dlPFC). The animals decided when and where to forage, based on whether their prediction of reward was fulfilled or violated. This prediction was not solely based on a history of reward delivery, but also on the understanding that waiting longer improves the chance of reward. The decoded reward dynamics were continuously represented in a subspace of the high-dimensional population activity, and predicted animals subsequent choice better than the true experimental variables and as well as the raw neural activity. Our results indicate that monkeys foraging strategy is based on a cortical model of reward dynamics as animals freely explore their environment.

neuroscience↗

bioRxiv2019

Tracking the mind’s eye: Primate gaze behavior during virtual visuomotor navigation reflects belief dynamics

To take the best actions, we often need to maintain and update beliefs about variables that cannot be directly observed. To understand the principles underlying such belief updates, we need tools to uncover subjects belief dynamics from natural behaviour. We tested whether eye movements could be used to infer subjects beliefs about latent variables using a naturalistic, visuomotor navigation task. We observed eye movements that appeared to continuously track the goal location even when no visible target was present there. Accurate goal-tracking was associated with improved task performance, and inhibiting eye movements in humans impaired navigation precision. By using passive stimulus playback and manipulating stimulus reliability, we show that subjects eye movements are likely voluntary, rather than reflexive. These results suggest that gaze dynamics play a key role in action-selection during challenging visuomotor behaviours, and may possibly serve as a window into the subjects dynamically evolving internal beliefs.

animal behavior and cognition↗

bioRxiv2018

Inception in visual cortex: in vivo-silico loops reveal most exciting images

Much of our knowledge about sensory processing in the brain is based on quasi-linear models and the stimuli that optimally drive them. However, sensory information processing is nonlinear, even in primary sensory areas, and optimizing sensory input is difficult due to the high-dimensional input space. We developed inception loops, a closed-loop experimental paradigm that combines in vivo recordings with in silico nonlinear response modeling to identify the Most Exciting Images (MEIs) for neurons in mouse V1. When presented back to the brain, MEIs indeed drove their target cells significantly better than the best stimuli identified by linear models. The MEIs exhibited complex spatial features that deviated from the textbook ideal of V1 as a bank of Gabor filters. Inception loops represent a widely applicable new approach to dissect the neural mechanisms of sensation.

neuroscience↗

bioRxiv2018

Dynamic brain interactions during picture naming

Brain computations involve multiple processes by which sensory information is encoded and transformed to drive behavior. These computations are thought to be mediated by dynamic interactions between populations of neurons. Here we demonstrate that human brains exhibit a reliable sequence of neural interactions during speech production. We use an autoregressive hidden Markov model to identify dynamical network states exhibited by electrocorticographic signals recorded from human neurosurgical patients. Our method resolves dynamic latent network states on a trial-by-trial basis. We characterize individual network states according to the patterns of directional information flow between cortical regions of interest. These network states occur consistently and in a specific, interpretable sequence across trials and subjects: a fixed-length visual processing state is followed by a variable-length language state, and then by a terminal articulation state. This empirical evidence validates classical psycholinguistic theories that have posited such intermediate states during speaking. It further reveals these state dynamics are not localized to one brain area or one sequence of areas, but are instead a network phenomenon.

neuroscience↗

bioRxiv2018

Stimulus domain transfer in recurrent models for large scale cortical population prediction on video

To better understand the representations in visual cortex, we need to generate better predictions of neural activity in awake animals presented with their ecological input: natural video. Despite recent advances in models for static images, models for predicting responses to natural video are scarce and standard linear-nonlinear models perform poorly. We developed a new deep recurrent network architecture that predicts inferred spiking activity of thousands of mouse V1 neurons simulta-neously recorded with two-photon microscopy, while accounting for confounding factors such as the animals gaze position and brain state changes related to running state and pupil dilation. Powerful system identification models provide an opportunity to gain insight into cortical functions through in silico experiments that can subsequently be tested in the brain. However, in many cases this approach requires that the model is able to generalize to stimulus statistics that it was not trained on, such as band-limited noise and other parameterized stimuli. We investigated these domain transfer properties in our model and find that our model trained on natural images is able to correctly predict the orientation tuning of neurons in responses to artificial noise stimuli. Finally, we show that we can fully generalize from movies to noise and maintain high predictive performance on both stimulus domains by fine-tuning only the final layers weights on a network otherwise trained on natural movies. The converse, however, is not true.

neuroscience↗

bioRxiv2017

A Dynamic Bayesian Observer Model Reveals Origins of Bias in Visual Path Integration

Path integration is a navigation strategy by which animals track their position by integrating their self-motion velocity over time. To identify the computational origins of bias in visual path integration, we asked human subjects to navigate in a virtual environment using optic flow, and found that they generally travelled beyond the goal location. Such a behaviour could stem from leaky integration of unbiased self-motion velocity estimates, or from a prior expectation favouring slower speeds that causes underestimation of velocity. We tested both alternatives using a probabilistic framework that maximizes expected reward, and found that subjects biases were better explained by a slow-speed prior than imperfect integration. When subjects integrate paths over long periods, this framework intriguingly predicts a distance-dependent bias reversal due to build-up of uncertainty, which we also confirmed experimentally. These results suggest that visual path integration performance is limited largely by biases in processing optic flow rather than by suboptimal signal integration.

neuroscience↗

bioRxiv2017

Inferring decoding strategies for multiple correlated neural populations

Studies of neuron-behaviour correlation and causal manipulation have long been used separately to understand the neural basis of perception. Yet these approaches sometimes lead to drastically conflicting conclusions about the functional role of brain areas. Theories that focus only on choice-related neuronal activity cannot reconcile those findings without additional experiments involving large-scale recordings to measure interneuronal correlations. By expanding current theories of neural coding and incorporating results from inactivation experiments, we demonstrate here that it is possible to infer decoding weights of different brain areas without precise knowledge of the correlation structure. We apply this technique to neural data collected from two different cortical areas in macaque monkeys trained to perform a heading discrimination task. We identify two opposing decoding schemes, each consistent with data depending on the nature of correlated noise. Our theory makes specific testable predictions to distinguish these scenarios experimentally without requiring measurement of the underlying noise correlations.\n\nAuthor SummaryThe neocortex is structurally organized into distinct brain areas. The role of specific brain areas in sensory perception is typically studied using two kinds of laboratory experiments: those that measure correlations between neural activity and reported percepts, and those that inactivate a brain region and measure the resulting changes in percepts. The two types of experiments have generally been interpreted in isolation, in part because no theory has been able combine their outcomes. Here, we describe a mathematical framework that synthesizes both kinds of results, giving us a new way to assess how different brain areas contribute to perception. When we apply our framework to experiments on behaving monkeys, we discover two models that can explain the perplexing finding that one brain area can predict an animals percepts, even though the percepts are not affected when that brain area is inactivated. The two models ascribe dramatically different efficiencies to brain computation. We show that these two models can be distinguished by an experiment that measures correlations while inactivating different brain areas.

neuroscience↗