Biology subjects

Paninski, L.

Publications and source records attributed to Paninski, L..

At least 37 records · Page 2Linked to original sources

bioRxiv2023

Bayesian target optimisation for high-precision holographic optogenetics

Two-photon optogenetics has transformed our ability to probe the structure and function of neural circuits. However, achieving precise optogenetic control of neural ensemble activity has remained fundamentally constrained by the problem of off-target stimulation (OTS): the inadvertent activation of nearby non-target neurons due to imperfect confinement of light onto target neurons. Here we propose a novel computational approach to this problem called Bayesian target optimisation. Our approach uses nonparametric Bayesian inference to model neural responses to optogenetic stimulation, and then optimises the laser powers and optical target locations needed to achieve a desired activity pattern with minimal OTS. We validate our approach in simulations and using data from in vitro experiments, showing that Bayesian target optimisation considerably reduces OTS across all conditions we test. Together, these results establish our ability to overcome OTS, enabling optogenetic stimulation with substantially improved precision.

neuroscience↗

bioRxiv2023

Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling, and cloud-native open-source tools

Contemporary pose estimation methods enable precise measurements of behavior via supervised deep learning with hand-labeled video frames. Although effective in many cases, the supervised approach requires extensive labeling and often produces outputs that are unreliable for downstream analyses. Here, we introduce "Lightning Pose," an efficient pose estimation package with three algorithmic contributions. First, in addition to training on a few labeled video frames, we use many unlabeled videos and penalize the network whenever its predictions violate motion continuity, multiple-view geometry, and posture plausibility (semi-supervised learning). Second, we introduce a network architecture that resolves occlusions by predicting pose on any given frame using surrounding unlabeled frames. Third, we refine the pose predictions post-hoc by combining ensembling and Kalman smoothing. Together, these components render pose trajectories more accurate and scientifically usable. We release a cloud application that allows users to label data, train networks, and predict new videos directly from the browser.

neuroscience↗

bioRxiv2022

Robust online multiband drift estimation in electrophysiology data

High-density electrophysiology probes have opened new possibilities for systems neuroscience in human and non-human animals, but probe motion (or drift) while recording poses a challenge for downstream analyses, particularly in human recordings. Here, we improve on the state of the art for tracking this drift with an algorithm termed DREDge (Decentralized Registration of Electrophysiology Data) with four major contributions. First, we extend previous decentralized methods to exploit multiband information, leveraging the local field potential (LFP), in addition to spikes detected from the action potentials (AP). Second, we show that the LFP-based approach enables registration at sub-second temporal resolution. Third, we introduce an efficient online motion tracking algorithm, allowing the method to scale up to longer and higher spatial resolution recordings, which could facilitate real-time applications. Finally, we improve the robustness of the approach by accounting for the nonstationarities that occur in real data and by automating parameter selection. Together, these advances enable fully automated scalable registration of challenging datasets from both humans and mice.

neuroscience↗

bioRxiv2022

Rapid learning of neural circuitry from holographic ensemble stimulation enabled by model-based compressed sensing

Discovering how neural computations are implemented in the cortex at the level of monosynaptic connectivity requires probing for the existence of synapses from possibly thousands of presynaptic candidate neurons. Two-photon optogenetics has been shown to be a promising technology for mapping such monosynaptic connections via serial stimulation of neurons with single-cell resolution. However, this approach is limited in its ability to uncover connectivity at large scales because stimulating neurons one-by-one requires prohibitively long experiments. Here we developed novel computational tools that, when combined, enable learning of monosynaptic connectivity from high-speed holographic neural ensemble stimulation. First, we developed a model-based compressed sensing algorithm that identifies connections from postsynaptic responses evoked by stimulation of many neurons at once, considerably increasing the rate at which the existence and strength of synapses are screened. We show that this model-based approach, explicitly incorporating known biophysics of optogenetic mapping experiments, is critical for accurately determining synaptic connectivity using compressed sensing. Second, we developed a deep learning method that isolates the postsynaptic response evoked by each stimulus, allowing stimulation to rapidly switch between ensembles without waiting for the postsynaptic response to return to baseline. We then validated our approach by performing large-scale connectivity mapping experiments in slices from layer 2/3 of mouse primary visual cortex. Together, our system increases the throughput of monosynaptic connectivity mapping by an order of magnitude over existing approaches, enabling the acquisition of connectivity maps at speeds needed to discover the synaptic circuitry implementing neural computations.

neuroscience↗

bioRxiv2022

Versatile Multiple Object Tracking in Sparse 2D/3D Videos Via Diffeomorphic Image Registration

Tracking body parts in behaving animals, extracting fluorescence signals from cells embedded in deforming tissue, and analyzing cell migration patterns during development all require tracking objects with partially correlated motion. As dataset sizes increase, manual tracking of objects becomes prohibitively inefficient and slow, necessitating automated and semi-automated computational tools. Unfortunately, existing methods for multiple object tracking (MOT) are either developed for specific datasets and hence do not generalize well to other datasets, or require large amounts of training data that are not readily available. This is further exacerbated when tracking fluorescent sources in moving and deforming tissues, where the lack of unique features and sparsely populated images create a challenging environment, especially for modern deep learning techniques. By leveraging technology recently developed for spatial transformer networks, we propose ZephIR, an image registration framework for semi-supervised MOT in 2D and 3D videos. ZephIR can generalize to a wide range of biological systems by incorporating adjustable parameters that encode spatial (sparsity, texture, rigidity) and temporal priors of a given data class. We demonstrate the accuracy and versatility of our approach in a variety of applications, including tracking the body parts of a behaving mouse and neurons in the brain of a freely moving C. elegans. We provide an open-source package along with a web-based graphical user interface that allows users to provide small numbers of annotations to interactively improve tracking results.

neuroscience↗

bioRxiv2022

Reproducibility of in-vivo electrophysiological measurements in mice

Understanding brain function relies on the collective work of many labs generating reproducible results. However, reproducibility has not been systematically assessed within the context of electrophysiological recordings during cognitive behaviors. To address this, we formed a multi-lab collaboration using a shared, open-source behavioral task and experimental apparatus. Experimenters in ten laboratories repeatedly targeted Neuropixels probes to the same location (spanning secondary visual areas, hippocampus, and thalamus) in mice making decisions; this generated a total of 121 experimental replicates, a unique dataset for evaluating reproducibility of electrophysiology experiments. Despite standardizing both behavioral and electrophysiological procedures, some experimental outcomes were highly variable. A closer analysis uncovered that variability in electrode targeting hindered reproducibility, as did the limited statistical power of some routinely used electrophysiological analyses, such as single-neuron tests of modulation by individual task parameters. Reproducibility was enhanced by histological and electrophysiological quality-control criteria. Our observations suggest that data from systems neuroscience is vulnerable to a lack of reproducibility, but that across-lab standardization, including metrics we propose, can serve to mitigate this.

neuroscience↗

bioRxiv2021

Three-dimensional spike localization and improved motion correction for Neuropixels recordings

Neuropixels (NP) probes are dense linear multi-electrode arrays that have rapidly become essential tools for studying the electrophysiology of large neural populations. Unfortunately, a number of challenges remain in analyzing the large datasets output by these probes. Here we introduce several new methods for extracting useful spiking information from NP probes. First, we use a simple point neuron model, together with a neural-network denoiser, to efficiently map single spikes detected on the probe into three-dimensional localizations. Previous methods localized individual spikes in two dimensions only; we show that the new localization approach is significantly more robust and provides an improved feature set for clustering spikes according to neural identity ("spike sorting"). Next, we denoise the resulting three-dimensional point-cloud representation of the data, and show that the resulting 3D images can be accurately registered over time, leading to improved tracking of time-varying neural activity over the probe, and in turn, crisper estimates of neural clusters over time. Open source code is available at https://github.com/int-brain-lab/spikes_localization_registration.git.

neuroscience↗

bioRxiv2021

Flygenvectors: The spatial and temporal structure of neural activity across the fly brain

What are the spatial and temporal scales of brainwide neuronal activity, and how do activities at different scales interact? We used SCAPE microscopy to image a large fraction of the central brain of adult Drosophila melanogaster with high spatiotemporal resolution while flies engaged in a variety of behaviors, including running, grooming and flailing. This revealed neural representations of behavior on multiple spatial and temporal scales. The activity of most neurons across the brain correlated (or, in some cases, anticorrelated) with running and flailing over timescales that ranged from seconds to almost a minute. Grooming elicited a much weaker global response. Although these behaviors accounted for a large fraction of neural activity, residual activity not directly correlated with behavior was high dimensional. Many dimensions of the residual activity reflect the activity of small clusters of spatially organized neurons that may correspond to genetically defined cell types. These clusters participate in the global dynamics, indicating that neural activity reflects a combination of local and broadly distributed components. This suggests that microcircuits with highly specified functions are provided with knowledge of the larger context in which they operate, conferring a useful balance of specificity and flexibility.

neuroscience↗

bioRxiv2021

Improved blind demixing methods for recovering dense neuronal morphology from barcode imaging data

Cellular barcoding methods offer the exciting possibility of infinite-pseudocolor anatomical reconstruction -- i.e., assigning each neuron its own random unique barcoded pseudocolor, and then using these pseudocolors to trace the microanatomy of each neuron. Here we use simulations, based on densely-reconstructed electron microscopy microanatomy, with signal structure matched to real barcoding data, to quantify the feasibility of this procedure. We develop a new blind demixing approach to recover the barcodes that label each neuron. We also develop a neural network which uses these barcodes to reconstruct the neuronal morphology from the observed fluorescence imaging data, connecting the dots between discontiguous amplicon signals. We find that accurate recovery should be feasible, provided that the barcode signal density is sufficiently high. This study suggests the possibility of mapping the morphology and projection pattern of many individual neurons simultaneously, at high resolution and at large scale, via conventional light microscopy.

neuroscience↗

bioRxiv2021

Semi-supervised sequence modeling for improved behavioral segmentation

A popular approach to quantifying animal behavior from video data is through discrete behavioral segmentation, wherein video frames are labeled as containing one or more behavior classes such as walking or grooming. Sequence models learn to map behavioral features extracted from video frames to discrete behaviors, and both supervised and unsupervised methods are common. However, each approach has its drawbacks: supervised models require a time-consuming annotation step where humans must hand label the desired behaviors; unsupervised models may fail to accurately segment particular behaviors of interest. We introduce a semi-supervised approach that addresses these challenges by constructing a sequence model loss function with (1) a standard supervised loss that classifies a sparse set of hand labels; (2) a weakly supervised loss that classifies a set of easy-to-compute heuristic labels; and (3) a self-supervised loss that predicts the evolution of the behavioral features. With this approach, we show that a large number of unlabeled frames can improve supervised segmentation in the regime of sparse hand labels and also show that a small number of hand labeled frames can increase the precision of unsupervised segmentation.

animal behavior and cognition↗

bioRxiv2021

Visualizing the organization and differentiation of the male-specific nervous system of C. elegans

Sex differences in the brain are prevalent throughout the animal kingdom and particularly well appreciated in the nematode C. elegans. While 294 neurons are shared between the two sexes, the nervous system of the male contains an additional 93 malespecific neurons, most of which have received very little attention so far. To make these neurons amenable for future study, we describe here how a multicolor, multipromoter reporter transgene, NeuroPAL, is capable of visualizing the distinct identities of all male specific neurons. We used this tool to visualize and characterize a number of features of the male-specific nervous system. We provide several proofs of concept for using NeuroPAL to identify the sites of expression of gfp-tagged reporter genes. We demonstrate the usage of NeuroPAL for cellular fate analysis by analyzing the effect of removal of developmental patterning genes, including a HOX cluster gene (egl-5), a miRNA (lin-4) and a proneural gene (lin-32/Ato), on neuronal identity acquisition within the male-specific nervous system. We use NeuroPAL and its intrinsic cohort of more than 40 distinct differentiation markers to show that, even though male-specific neurons are generated throughout all four larval stages, they execute their terminal differentiation program in a coordinated manner in the fourth larval stage that is concomitant with male tale retraction. This wave of differentiation couples neuronal maturation programs with the appearance of sexual organs. We call this wave "just-in-time" differentiation by its analogy to the mechanism of "just-in-time" transcription of metabolic pathway genes.

neuroscience↗

bioRxiv2021

Partitioning variability in animal behavioral videos using semi-supervised variational autoencoders

Recent neuroscience studies demonstrate that a deeper understanding of brain function requires a deeper understanding of behavior. Detailed behavioral measurements are now often collected using video cameras, resulting in an increased need for computer vision algorithms that extract useful information from video data. Here we introduce a new video analysis tool that combines the output of supervised pose estimation algorithms (e.g. DeepLabCut) with unsupervised dimensionality reduction methods to produce interpretable, low-dimensional representations of behavioral videos that extract more information than pose estimates alone. We demonstrate this tool by extracting interpretable behavioral features from videos of three different head-fixed mouse preparations, as well as a freely moving mouse in an open field arena, and show how these interpretable features can facilitate downstream behavioral and neural analyses. We also show how the behavioral features produced by our model improve the precision and interpretation of these downstream analyses compared to using the outputs of either fully supervised or fully unsupervised methods alone.

neuroscience↗

bioRxiv2020

Recurrent Switching Dynamical Systems Models for Multiple Interacting Neural Populations

Modern recording techniques can generate large-scale measurements of multiple neural populations over extended time periods. However, it remains a challenge to model non-stationary interactions between high-dimensional populations of neurons. To tackle this challenge, we develop recurrent switching linear dynamical systems models for multiple populations. Here, each high-dimensional neural population is represented by a unique set of latent variables, which evolve dynamically in time. Populations interact with each other through this low-dimensional space. We allow the nature of these interactions to change over time by using a discrete set of dynamical states. Additionally, we parameterize these discrete state transition rules to capture which neural populations are responsible for switching between interaction states. To fit the model, we use variational expectation-maximization with a structured mean-field approximation. After validating the model on simulations, we apply it to two different neural datasets: spiking activity from motor areas in a non-human primate, and calcium imaging from neurons in the nematode C. elegans. In both datasets, the model reveals behaviorally-relevant discrete states with unique inter-population interactions and different populations that predict transitioning between these states.

neuroscience↗

bioRxiv2020

Multiscale and multimodal reconstruction of cortical structure and function

We present a semi-automated reconstruction of L2/3 mouse primary visual cortex from 3 million cubic microns of electron microscopic images, including pyramidal and inhibitory neurons, astrocytes, microglia, oligodendrocytes and precursors, pericytes, vasculature, mitochondria, and synapses. Visual responses of a subset of pyramidal cells are included. The data are being made publicly available, along with tools for programmatic and 3D interactive access. The density of synaptic inputs onto inhibitory neurons varies across cell classes and compartments. We uncover a compartment-specific correlation between mitochondrial coverage and synapse density. Frequencies of connectivity motifs in the graph of pyramidal cells are predicted quite accurately from node degrees using the configuration model of random graphs. Cells receiving more connections from nearby cells exhibit stronger and more reliable visual responses. These example findings illustrate the resources utility for relating structure and function of cortical circuits as well as for neuronal cell biology.

neuroscience↗

bioRxiv2020

Nonlinear decoding of natural images from large-scale primate retinal ganglion recordings

Decoding sensory stimuli from neural activity can provide insight into how the nervous system might interpret the physical environment, and facilitates the development of brain-machine interfaces. Nevertheless, the neural decoding problem remains a significant open challenge. Here, we present an efficient nonlinear decoding approach for inferring natural scene stimuli from the spiking activities of retinal ganglion cells (RGCs). Our approach uses neural networks to improve upon existing decoders in both accuracy and scalability. Trained and validated on real retinal spike data from > 1000 simultaneously recorded macaque RGC units, the decoder demonstrates the necessity of nonlinear computations for accurate decoding of the fine structures of visual stimuli. Specifically, high-pass spatial features of natural images can only be decoded using nonlinear techniques, while low-pass features can be extracted equally well by linear and nonlinear methods. Together, these results advance the state of the art in decoding natural stimuli from large populations of neurons. Author summaryNeural decoding is a fundamental problem in computational and statistical neuroscience. There is an enormous literature on this problem, applied to a wide variety of brain areas and nervous systems. Here we focus on the problem of decoding visual information from the retina. The bulk of previous work here has focused on simple linear decoders, applied to modest numbers of simultaneously recorded cells, to decode artificial stimuli. In contrast, here we develop a scalable nonlinear decoding method to decode natural images from the responses of over a thousand simultaneously recorded units, and show that this decoder significantly improves on the state of the art.

neuroscience↗

bioRxiv2020

Deep Graph Pose: a semi-supervised deep graphical model for improved animal pose tracking

Noninvasive behavioral tracking of animals is crucial for many scientific investigations. Recent transfer learning approaches for behavioral tracking have considerably advanced the state of the art. Typically these methods treat each video frame and each object to be tracked independently. In this work, we improve on these methods (particularly in the regime of few training labels) by leveraging the rich spatiotemporal structures pervasive in behavioral video -- specifically, the spatial statistics imposed by physical constraints (e.g., paw to elbow distance), and the temporal statistics imposed by smoothness from frame to frame. We propose a probabilistic graphical model built on top of deep neural networks, Deep Graph Pose (DGP), to leverage these useful spatial and temporal constraints, and develop an efficient structured variational approach to perform inference in this model. The resulting semi-supervised model exploits both labeled and unlabeled frames to achieve significantly more accurate and robust tracking while requiring users to label fewer training frames. In turn, these tracking improvements enhance performance on downstream applications, including robust unsupervised segmentation of behavioral "syllables," and estimation of interpretable "disentangled" low-dimensional representations of the full behavioral video. Open source code is available at https://github.com/paninski-lab/deepgraphpose.

animal behavior and cognition↗

bioRxiv2020

BARcode DEmixing through Non-negative Spatial Regression (BarDensr)

Modern spatial transcriptomics methods can target thousands of different types of RNA transcripts in a single slice of tissue. Many biological applications demand a high spatial density of transcripts relative to the imaging resolution, leading to partial mixing of transcript rolonies in many voxels; unfortunately, current analysis methods do not perform robustly in this highly-mixed setting. Here we develop a new analysis approach, BARcode DEmixing through Non-negative Spatial Regression (BarDensr): we start with a generative model of the physical process that leads to the observed image data and then apply sparse convex optimization methods to estimate the underlying (demixed) rolony densities. We apply Bar-Densr to simulated and real data and find that it achieves state of the art signal recovery, particularly in densely-labeled regions or data with low spatial resolution. Finally, BarDensr is fast and parallelizable. We provide open-source code as well as an implementation for the NeuroCAAS cloud platform. Author SummarySpatial transcriptomics technologies allow us to simultaneously detect multiple molecular targets in the context of intact tissues. These experiments yield images that answer two questions: which kinds of molecules are present, and where are they located in the tissue? In many experiments (e.g., mapping RNA expression in fine neuronal processes), it is desirable to increase the signal density relative to the imaging resolution. This may lead to mixing of signals from multiple RNA molecules into single imaging voxels; thus we need to demix the signals from these images. Here we introduce BarDensr, a new computational method to perform this demixing. The method is based on a forward model of the imaging process, followed by a convex optimization approach to approximately invert mixing induced during imaging. This new approach leads to significantly improved performance in demixing imaging data with dense expression and/or low spatial resolution.

genomics↗

bioRxiv2020

Extracting neural signals from semi-immobilized animals with deformable non-negative matrix factorization

Extracting calcium traces from populations of neurons is a critical step in the study of the large-scale neural dynamics that govern behavior. Accurate activity extraction requires the correction of motion and movement-induced deformations as well as demixing of signals that may overlap spatially due to limitations in optical resolution. Traditionally, non-negative matrix factorization (NMF) methods have been successful in demixing and denoising cellular calcium activity in relatively motionless or pre-registered videos. However, standard NMF methods fail in animals undergoing significant non-rigid motion; similarly, standard image registration methods based on template matching can fail when large changes in activity lead to mismatches with the image template. To address these issues simultaneously, we introduce a deformable non-negative matrix factorization (dNMF) framework that jointly optimizes registration with signal demixing. On simulated data and real semi-immobilized C. elegans microscopy videos, dNMF outperforms traditional demixing methods that account for motion and demixing separately. Finally, following the extraction of neural traces from multiple imaging experiments, we develop a quantile regression time-series normalization technique to account for varying neural signal intensity baselines across different animals or different imaging setups. Open source code implementing this pipeline is available at https://github.com/amin-nejat/dNMF.

neuroscience↗