Biology subjects

Butts, D. A.

Publications and source records attributed to Butts, D. A..

11 recordsLinked to original sources

bioRxiv2026

Active sensing explains cortical variability and improves visual information

Active vision is often considered at the scale of saccades, with each fixation treated as a static visual sample. Yet the eyes continue to move during fixation, continually transforming the retinal input. Here we measured these movements at high precision while recording neuronal populations in the foveal primary visual cortex of awake marmosets viewing repeated flashed natural images. Fixational eye movements accounted for most conditional firing-rate variability and nearly all shared variability not explained by the trial-averaged response. An image-computable model captured this modulation largely from gaze-dependent retinal input; explicit eye-position and eye-velocity signals added little predictive power. Under natural viewing, retinal motion redistributed image power across temporal frequencies, and motion effects grew when this power engaged spatiotemporal tuning. In model-derived populations, fixational motion increased the spatial information carried per spike relative to stabilized input, with sharpening growing across successive nonlinear stages. Thus, activity commonly assigned to cortical noise is not just structured visual reafference generated by the movement of the observer, but is beneficial signal which sharpens spatial representations.

neuroscience↗

bioRxiv2023

Hierarchical VAEs provide a normative account of motion processing in the primate brain

The relationship between perception and inference, as postulated by Helmholtz in the 19th century, is paralleled in modern machine learning by generative models like Variational Autoencoders (VAEs) and their hierarchical variants. Here, we evaluate the role of hierarchical inference and its alignment with brain function in the domain of motion perception. We first introduce a novel synthetic data framework, Retinal Optic Flow Learning (ROFL), which enables control over motion statistics and their causes. We then present a new hierarchical VAE and test it against alternative models on two downstream tasks: (i) predicting ground truth causes of retinal optic flow (e.g., self-motion); and (ii) predicting the responses of neurons in the motion processing pathway of primates. We manipulate the model architectures (hierarchical versus non-hierarchical), loss functions, and the causal structure of the motion stimuli. We find that hierarchical latent structure in the model leads to several improvements. First, it improves the linear decodability of ground truth factors and does so in a sparse and disentangled manner. Second, our hierarchical VAE outperforms previous state-of-the-art models in predicting neuronal responses and exhibits sparse latent-to-neuron relationships. These results depend on the causal structure of the world, indicating that alignment between brains and artificial neural networks depends not only on architecture but also on matching ecologically relevant stimulus statistics. Taken together, our results suggest that hierarchical Bayesian inference underlines the brains understanding of the world, and hierarchical VAEs can effectively model this understanding.

neuroscience↗

bioRxiv2022

Activity in primate visual cortex is minimally driven by spontaneous movements

Organisms process sensory information in the context of their own moving bodies, an idea referred to as embodiment. This idea is important for developmental neuroscience, and increasingly plays a role in robotics and systems neuroscience. The mechanisms that support such embodiment are unknown, but a manifestation could be the observation in mice of brain-wide neuromodulation, including in the primary visual cortex, driven by task-irrelevant spontaneous body movements. Here we tested this hypothesis in macaque monkeys, a primate model for human vision, by simultaneously recording visual cortex activity and facial and body movements. Activity in the visual cortex (V1, V2, V3/V3A) was associated with the animals own movements, but this modulation was largely explained by the impact of the movements on the retinal image. These results suggest that embodiment in primate vision may be realized by input provided by the eyes themselves.

neuroscience↗

bioRxiv2022

Running modulates primate and rodent visual cortex via common mechanism but quantitatively distinct implementation

When mice run, activity in their primary visual cortex (V1) is strongly modulated. This observation has altered conceptions of a brain region assumed to be a passive image processor. Extensive work has followed to dissect the circuits and functions of running-correlated modulation. However, it remains unclear whether visual processing in primates might similarly change during locomotion. We therefore measured V1 activity in marmosets while they viewed stimuli on a treadmill. In contrast to mouse, running-correlated modulations of marmoset V1 were small, and tended to be slightly suppressive. Population-level analyses revealed trial- to-trial fluctuations of shared gain across V1 in both species, but while strongly correlated with running in mice, gain modulations were smaller and more often negatively correlated with running in marmosets. Thus, population-wide fluctuations of V1 may reflect a common feature of mammalian visual cortical function, but important quantitative differences point to distinct consequences for the relation between vision and action in primates versus rodents.

neuroscience↗

bioRxiv2021

Beyond Fixation: detailed characterization of neural selectivity in free-viewing primates

Virtually all vision studies use a fixation point to stabilize gaze, rendering stimuli on video screens fixed to retinal coordinates. This approach requires trained subjects, is limited by the accuracy of fixational eye movements, and ignores the role of eye movements in shaping visual input. To overcome these limitations, we developed a suite of hardware and software tools to study vision during natural behavior in untrained subjects. We show this approach recovers receptive fields and tuning properties of visual neurons from multiple cortical areas of marmoset monkeys. Combined with high-precision eye-tracking, it achieves sufficient resolution to recover the receptive fields of foveal V1 neurons. These findings demonstrate the power of free viewing to characterize neural response while simultaneously studying the dynamics of natural behavior. HighlightsO_LIWe introduce a free-viewing paradigm for studying neural mechanisms of visual processing during active vision C_LIO_LIReceptive fields (RFs) and neural selectivity in primary visual cortex (V1) and area MT can be extracted during free-viewing in minimally-trained subjects C_LIO_LINovel high-resolution eye tracking in this context supports detailed measurements of receptive fields in foveal V1 C_LI

neuroscience↗

bioRxiv2021

Model-based characterization of the selectivity of neurons in primary visual cortex

To understand the complexity of stimulus selectivity in primary visual cortex (V1), models constructed to match observed responses to complex time-varying stimuli, instead of to explain responses to simple parametric stimuli, are increasingly used. While such models often can more accurately reflect the computations performed by V1 neurons in more natural visual environments, they do not by themselves provide insight into established measures of V1 neural selectivity such as receptive field size, spatial frequency tuning and phase invariance. Here, we suggest a series of analyses that can be directly applied to encoding models to link complex encoding models to more interpretable aspects of stimulus selectivity, applied to nonlinear models of V1 neurons recorded in awake macaque in response to random bar stimuli. In linking model properties to more classical measurements, we demonstrate several novel aspects of V1 selectivity not available to simpler experimental measurements. For example, we find that individual spatiotemporal elements of the V1 models often have a smaller spatial scale than the overall neuron sensitivity, and that this results in non-trivial tuning to spatial frequencies. Additionally, our proposed measures of nonlinear integration suggest that more classical classifications of V1 neurons into simple versus complex cells are spatial-frequency dependent. In total, rather than obfuscate classical characterizations of V1 neurons, model-based characterizations offer a means to more fully understand their selectivity, and provide a means to link their classical tuning properties to their roles in more complex, natural, visual processing. Significance statementVisual neurons are increasingly being studied with more complex, natural visual stimuli, with increasingly complex models necessary to characterize their response properties. Here, we describe a battery of analyses that relate these more complex models to classical characterizations. Using such model-based characterizations of V1 neurons furthermore yields several new insights into V1 processing not possible to capture in more classical means to measure their visual selectivity.

neuroscience↗

bioRxiv2020

A latent variable approach to decoding neural population activity

Decoding is a powerful approach for measuring the information contained in the activity of neural populations. As a result, decoding analyses are now used across a wide range of model organisms and experimental paradigms. However, typical analyses employ general purpose decoding algorithms that do not explicitly take advantage of the structure of neural variability, which is often low-dimensional and can thus be effectively characterized using latent variables. Here we propose a new decoding framework that exploits the low-dimensional structure of neural population variability by removing correlated variability that is unrelated to the decoded variable, then decoding the resulting denoised activity. We demonstrate the efficacy of this framework using simulated data, where the true upper bounds for decoding performance are known. A linear version of our decoder provides an estimator for the decoded variable that can be more efficient than other commonly used linear estimators such as linear discriminant analysis. In addition, our proposed decoding framework admits a simple extension to nonlinear decoding that compares favorably to standard feed-forward neural networks. By explicitly modeling shared population variability, the success of the resulting linear and nonlinear decoders also offers a new perspective on the relationship between shared variability and information contained in large neural populations.

neuroscience↗

bioRxiv2018

Current models cannot account for V1’s specialisation for binocular natural image statistics

A long-standing observation about primary visual cortex (V1) is that the stimulus selectivity of neurons can be well explained with a cascade of linear computations followed by a nonlinear rectification stage. This framework remains highly influential in systems neuroscience and has also inspired recent efforts in artificial intelligence. The success of these models include describing the disparity-selectivity of binocular neurons in V1. Some aspects of real neuronal disparity responses are hard to explain with simple linear-nonlinear models, notably the attenuated response of real cells to \"anticorrelated\" stimuli which violate natural binocular image statistics. General linear-nonlinear models can account for this attenuation, but no one has yet tested whether they quantitatively match the response of real neurons. Here, we exhaustively test this framework using recently developed optimisation techniques. We show that many cells are very poorly characterised by even general linear-nonlinear models. Strikingly, the models can account for neuronal responses to unnatural anticorrelated stimuli as well as to most natural, correlated stimuli. However, the models fail to capture the particularly strong response to binocularly correlated stimuli at the preferred disparity of the cell. Thus, V1 neurons perform an amplification of responses to correlated stimuli which cannot be accounted for by a linear-nonlinear cascade. The implication is that even simple stimulus selectivity in V1 requires more complex computations than previously envisaged.\n\nSignificance statementA long-standing question in sensory systems neuroscience is whether the computations performed by neurons in primary visual cortex can be described by repeated elements of linear-nonlinear units (a linear filtering/pooling stage followed by a subsequent output nonlinearity, such as a squaring). This question goes back to the Nobel-prize winning work by Hubel & Wiesel who argued that orientation selectivity in V1 can qualitatively be explained in this way. In this paper, we show that V1 neurons have an amplification of their response to stimuli which are contrast matched in the two eyes, and that the recovered models cannot describe this property. We argue that this likely represents more sophisticated computations than can be compactly described by the linear-nonlinear cascade framework.

neuroscience↗

bioRxiv2018

Functional characterization of retinal ganglion cells using tailored nonlinear modeling

There are 20-50 functionally- and anatomically-distinct ganglion cell types in the mammalian retina; each type encodes a unique feature of the visual world and conveys it via action potentials to the brain. Individual ganglion cells receive input from unique presynaptic retinal circuits, and the characteristic patterns of light-evoked action potentials in each ganglion cell type therefore reflect computations encoded in synaptic input and in postsynaptic signal integration and spike generation. Unfortunately, there is a dearth of tools for characterizing retinal ganglion cell computation. Therefore, we developed a statistical model, the separable Nonlinear Input Model, capable of characterizing the large array of distinct computations reflected in retinal ganglion cell spiking. We recorded ganglion cell responses to a correlated noise (\"cloud\") stimulus designed to accentuate the important features of retinal processing in an in vitro preparation of mouse retina and found that this model accurately predicted ganglion cell responses at high spatiotemporal resolution. It identified multiple receptive fields (RFs) reflecting the main excitatory and suppressive components of the response of each neuron. Most significantly, our model succeeds where others fail, accurately identifying ON-OFF cells and segregating their distinct ON and OFF selectivity and demonstrating the presence of different types of suppressive receptive fields. In total, our computational approach offers rich description of ganglion cell computation and sets a foundation for relating retinal computation to retinal circuitry.

neuroscience↗

bioRxiv2018

Characterizing the nonlinear structure of shared variability in cortical neuron populations using neural networks

Sensory neurons often have variable responses to repeated presentations of the same stimulus, which can significantly degrade the information contained in those responses. Such variability is often shared across many neurons, which in principle can allow a decoder to mitigate the effects of such noise, depending on the structure of the shared variability and its relationship to sensory encoding at the population level. Latent variable models offer an approach for characterizing the structure of this shared variability in neural population recordings, although they have thus far typically been used under restrictive mathematical assumptions, such as assuming linear transformations between the latent variables and neural activity. Here we leverage recent advances in machine learning to introduce two nonlinear latent variable models for analyzing large-scale neural recordings. We first present a general nonlinear latent variable model that is agnostic to the stimulus tuning properties of the individual neurons, and is hence well suited for exploring neural populations whose tuning properties are not well characterized. This motivates a second class of model, the Generalized Affine Model, which simultaneously determines each neurons stimulus selectivity and a set of latent variables that modulate these stimulus responses both additively and multiplicatively. While these approaches can detect general nonlinear relationships in shared neural variability, we find that neural activity recorded in anesthetized primary visual cortex (V1) is best described by a single additive and single multiplicative latent variable, i.e., an \"affine model\". In contrast, application of the same models to recordings in awake macaque prefrontal cortex discover more general nonlinearities to compactly describe the population response variability. These results thus demonstrate how nonlinear latent variable models can be used to describe population variability, and suggest that a range of methods is necessary to study different brain regions under different experimental conditions.

neuroscience↗

bioRxiv2017

Spatial organization of the mouse auditory cortex to sound dynamics revealed using automated image segmentation

Sound stimuli are characterized by their rich spectral and temporal dynamic properties. Individual neurons in auditory cortex (ACX) encode both spectral and temporal aspects of sounds e.g. sound onset and/or offset. While the different fields of the ACX show gradients of frequency selectivity the large-scale organization of sound dynamics is unknown. We used widefield imaging of GCaMP6s in awake mouse ACX combined with a novel unsupervised image segmentation technique to investigate the spatiotemporal representation of sound onset and offset. Using this technique, we identified known auditory fields but also detected novel ACX areas. Furthermore, we found that ACX areas differed in their responses to tone onset and offset. Multiple areas were preferentially activated by tone offset, and on-response areas were more spatially localized than off-response areas. We also found tonotopy in off-responses. Together our results demonstrate a different spatial distribution of neurons across ACX for processing sound onsets versus offsets.

neuroscience↗