Biology subjects

Tolias, A. S.

Publications and source records attributed to Tolias, A. S..

At least 19 recordsLinked to original sources

bioRxiv2026

A hippocampal digital Twin reveals latent structure in freely flying bats

Concurrent advances in artificial intelligence (AI), large-scale neuronal recording, and behavioral tracking have enabled increasingly powerful predictive models of brain-behavior relationships. Yet predictive accuracy does not automatically yield interpretable principles of neural organization, particularly during natural behavior, where the relevant variables computed are often unknown in advance. Here, we developed a functional digital twin of the hippocampus, trained jointly on large-scale neural activity and self-directed three-dimensional flight in freely flying bats. A multimodal, multitask transformer accurately predicted both neural activity and behavior, outperformed standard approaches, and generalized across recording sessions. Interrogating the model's representations suggested a form of hippocampal generalization across self-selected trajectories that was not apparent in physical coordinates, but was plainly visible in the model's low-dimensional latent space. We confirmed this organization directly in the recorded neural activity: neurons fired at corresponding stages of spatially distinct paths connecting the same origin and destination, preserving their relative firing order even as their firing fields shifted with the path. Analysis of neuron-specific readouts further identified functional relationships among neurons that generalized from flight to waking rest and sleep, brain states never used for training, and differed systematically between the hippocampal subregions CA1 and CA3. Together, these results show how AI-based functional digital twins can complement conventional analysis by generating testable hypotheses about neural organization during natural behavior.

neuroscience↗

bioRxiv2026

Visual uncertainty and task demands shape active sensing strategies in mice

In natural environments, animals actively sample visual information to guide behavior. Sensory feedback is dynamic and often requires active movements, whether saccading across the lines of this page or walking through a park. From high-acuity vision in hawks to low-acuity mice, many animals actively navigate to seek information, which can be called infotaxis. Although mice have relatively low-acuity vision, they still rely on sight for critical behaviors including navigation and prey capture. Yet, how sensitive they are to visual information and performing infotaxis has not been established. Here, we develop a virtual reality object discrimination task to investigate visual decision-making under naturalistic conditions. We show that mice perform infotaxis by actively seeking out informative views to guide their choices. Stimulus manipulations confirm that this strategy is modulated by the amount of available visual information. These results reveal that mice use principled active strategies to resolve visual uncertainty, highlighting a key role for information-seeking in natural vision. HighlightsO_LIMice perform active visual sensing in a free-range VR object discrimination task. C_LIO_LIThey perform infotaxis and select the correct object under variable visibility. C_LIO_LIDecisions decoded from head-body movements and speed reflect occlusion sensitivity. C_LI

neuroscience↗

bioRxiv2025

Beyond retinotopy: exploiting native visual representations in cortical neuroprostheses for vision loss remediation

Cortical prosthetic systems offer a promising path to restoring vision in blindness by stimulating neurons in the visual cortex to evoke visual percepts. To effectively encode visual information, however, it is essential to target neurons based on their functional encoding properties. Current stimulation protocols focus on retinotopic information, ignoring other key encoding properties, and therefore fail to reproduce complex visual percepts. We demonstrate that incorporating orientation selectivity alongside retinotopy to guide stimulation dramatically improves fidelity of the evoked activity to the underlying neural code. We propose a Bottlenecked Rotation-Equivariant CNN (BRCNN) and demonstrate that neural responses can be predicted to a large degree using only retinotopy and orientation preference. Using this model, we design a retinotopy-and-orientation aware stimulation protocol and validate it in a state-of-the-art large-scale optogenetic stimulation simulation framework of primary visual cortex. Our protocol elicits neural activity patterns exhibiting substantially higher correlation with natural vision responses compared to retinotopic-only approaches.

neuroscience↗

bioRxiv2023

Asymmetric distribution of color-opponent response types across mouse visual cortex supports superior color vision in the sky

Color is an important visual feature that informs behavior, and the retinal basis for color vision has been studied across various vertebrate species. While many studies have investigated how color information is processed in visual brain areas of primate species, we have limited understanding of how it is organized beyond the retina in other species, including most dichromatic mammals. In this study, we systematically characterized how color is represented in the primary visual cortex (V1) of mice. Using large-scale neuronal recordings and a luminance and color noise stimulus, we found that more than a third of neurons in mouse V1 are color-opponent in their receptive field center, while the receptive field surround predominantly captures luminance contrast. Furthermore, we found that color-opponency is especially pronounced in posterior V1 that encodes the sky, matching the statistics of natural scenes experienced by mice. Using unsupervised clustering, we demonstrate that the asymmetry in color representations across cortex can be explained by an uneven distribution of green-On/UV-Off color-opponent response types that are represented in the upper visual field. Finally, a simple model with natural scene-inspired parametric stimuli shows that green-On/UV-Off color-opponent response types may enhance the detection of "predatory"-like dark UV-objects in noisy daylight scenes. The results from this study highlight the relevance of color processing in the mouse visual system and contribute to our understanding of how color information is organized in the visual hierarchy across species.

neuroscience↗

bioRxiv2023

Energy Guided Diffusion for Generating Neurally Exciting Images

In recent years, most exciting inputs (MEIs) synthesized from encoding models of neuronal activity have become an established method to study tuning properties of biological and artificial visual systems. However, as we move up the visual hierarchy, the complexity of neuronal computations increases. Consequently, it becomes more challenging to model neuronal activity, requiring more complex models. In this study, we introduce a new attention readout for a convolutional data-driven core for neurons in macaque V4 that outperforms the state-of-the-art task-driven ResNet model in predicting neuronal responses. However, as the predictive network becomes deeper and more complex, synthesizing MEIs via straightforward gradient ascent (GA) can struggle to produce qualitatively good results and overfit to idiosyncrasies of a more complex model, potentially decreasing the MEIs model-to-brain transferability. To solve this problem, we propose a diffusion-based method for generating MEIs via Energy Guidance (EGG). We show that for models of macaque V4, EGG generates single neuron MEIs that generalize better across architectures than the state-of-the-art GA while preserving the within-architectures activation and requiring 4.7x less compute time. Furthermore, EGG diffusion can be used to generate other neurally exciting images, like most exciting natural images that are on par with a selection of highly activating natural images, or image reconstructions that generalize better across architectures. Finally, EGG is simple to implement, requires no retraining of the diffusion model, and can easily be generalized to provide other characterizations of the visual system, such as invariances. Thus EGG provides a general and flexible framework to study coding properties of the visual system in the context of natural images.1

neuroscience↗

bioRxiv2023

Deep learning-driven characterization of single cell tuning in primate visual area V4 unveils topological organization

Deciphering the brains structure-function relationship is key to understanding the neuronal mechanisms underlying perception and cognition. The cortical column, a vertical organization of neurons with similar functions, is a classic example of primate neocortex structure-function organization. While columns have been identified in primary sensory areas using parametric stimuli, their prevalence across higher-level cortex is debated, particularly regarding complex tuning in natural image space. However, a key hurdle in identifying columns is characterizing the complex, nonlinear tuning of neurons to high-dimensional sensory inputs. Building on prior findings of topological organization for features like color and orientation, we investigate functional clustering in macaque visual area V4 in non-parametric natural image space, using large-scale recordings and deep learning-based analysis. We combined linear probe recordings with deep learning methods to systematically characterize the tuning of >1,200 V4 neurons using in silico synthesis of most exciting images (MEIs), followed by in vivo verification. Single V4 neurons exhibited MEIs containing complex features, including textures and shapes, and even high-level attributes with eye-like appearance. Neurons recorded on the same silicon probe, inserted orthogonal to the cortical surface, often exhibited similarities in their spatial feature selectivity, suggesting a degree of functional organization along the cortical depth. We quantified MEI similarity using human psychophysics and distances in a contrastive learning-derived embedding space. Moreover, the selectivity of the V4 neuronal population showed evidence of clustering into functional groups of shared feature selectivity. These functional groups showed parallels with the feature maps of units in artificial vision systems, suggesting potential shared encoding strategies. These results demonstrate the feasibility and scalability of deep learning-based functional characterization of neuronal selectivity in naturalistic visual contexts, offering a framework for quantitatively mapping cortical organization across multiple levels of the visual hierarchy.

neuroscience↗

bioRxiv2023

Towards a Foundation Model of the Mouse Visual Cortex

The complexity of neural circuits makes it challenging to decipher the brains algorithms of intelligence. Recent break-throughs in deep learning have produced models that accurately simulate brain activity, enhancing our understanding of the brains computational objectives and neural coding. However, these models struggle to generalize beyond their training distribution, limiting their utility. The emergence of foundation models, trained on vast datasets, has introduced a new AI paradigm with remarkable generalization capabilities. We collected large amounts of neural activity from visual cortices of multiple mice and trained a foundation model to accurately predict neuronal responses to arbitrary natural videos. This model generalized to new mice with minimal training and successfully predicted responses across various new stimulus domains, such as coherent motion and noise patterns. It could also be adapted to new tasks beyond neural prediction, accurately predicting anatomical cell types, dendritic features, and neuronal connectivity within the MICrONS functional connectomics dataset. Our work is a crucial step toward building foundation brain models. As neuroscience accumulates larger, multi-modal datasets, foundation models will uncover statistical regularities, enabling rapid adaptation to new tasks and accelerating research.

neuroscience↗

bioRxiv2023

Bipartite invariance in mouse primary visual cortex

A primary goal of sensory systems is to extract robust and meaningful features that are invariant to variations in the sensory input. Characterizing these invariances at the neuronal level is crucial for understanding how the visual system supports generalization, but the high-dimensional nature of ecological stimuli poses major challenges. Consequently, our understanding of how the brain represents invariances has historically depended on a few examples, such as phase invariance to grating stimuli in V1 complex cells. Here, we leverage the inception loop paradigm --iterating between large-scale recordings, deep learning neuronal predictive models, and in silico experiments with in vivo verification--to characterize neuronal invariances in mouse V1. Using a neuronal predictive model, we synthesized Diverse Exciting Inputs (DEIs) that strongly drive target neurons while differing substantially in image space. These DEIs revealed a novel bipartite invariance: one portion of the receptive field encodes shift-invariant, high-frequency textures, while the other encodes a fixed, low-frequency spatial pattern. This subfield division aligned with object boundaries defined by spatial frequency differences in highly activating stimuli, suggesting bi-partite invariance contributes to segmentation. Our analysis of computational models and anatomical data from the MICrONS dataset revealed a hierarchical organization of excitatory neurons in mouse V1 Layers 2/3: We found that postsynaptic neurons exhibited greater invariance than their presynaptic inputs, while neurons with lower invariance formed more connections. These findings suggest a synaptic-level hierarchy that progressively increases neural invariance within the primary visual cortex. Intriguingly, similar high-low frequency bipartite patterns strongly activate certain units in artificial neural networks, suggesting that universal visual representations govern both biological and artificial systems, potentially aiding in the extraction of visual features from complex backgrounds.

neuroscience↗

bioRxiv2023

Pattern completion and disruption characterize contextual modulation in mouse visual cortex

Vision is fundamentally context-dependent, with neuronal responses influenced not just by local features but also by surrounding contextual information. In the visual cortex, studies using simple grating stimuli indicate that congruent stimuli--where the center and surround share the same orientation--are more inhibitory than when orientations are orthogonal, potentially serving redundancy reduction and predictive coding. Understanding these center-surround interactions in relation to natural image statistics is challenging due to the high dimensionality of the stimulus space, yet crucial for deciphering the neuronal code of real-world sensory processing. Utilizing large-scale recordings from mouse V1, we trained convolutional neural networks (CNNs) to predict and synthesize surround patterns that either optimally suppressed or enhanced responses to center stimuli, confirmed by in v ivo experiments. Contrary to the notion that congruent stimuli are suppressive, we found that surrounds that completed patterns based on natural image statistics were facilitatory, while disruptive surrounds were suppressive. Applying our CNN image synthesis method in macaque V1, we discovered that pattern completion within the near surround occurred more frequently with excitatory than with inhibitory surrounds, suggesting that our results in mice are conserved in macaques. Further, experiments and model analyses confirmed previous studies reporting the opposite effect with grating stimuli in both species. Using the MICrONS functional connectomics dataset, we observed that neurons with similar feature selectivity formed excitatory connections regardless of their receptive field overlap, aligning with the pattern completion phenomenon observed for excitatory surrounds. Finally, our empirical results emerged in a normative model of perception implementing Bayesian inference, where neuronal responses are modulated by prior knowledge of natural scene statistics. In summary, our findings identify a novel relationship between contextual information and natural scene statistics and provide evidence for a role of contextual modulation in hierarchical inference.

neuroscience↗

bioRxiv2023

Combined statistical-mechanistic modeling links ionchannel genes to physiology of cortical neuron types

Neural cell types have classically been characterized by their anatomy and electrophysiology. More recently, single-cell transcriptomics has enabled an increasingly fine genetically defined taxonomy of cortical cell types, but the link between the gene expression of individual cell types and their physiological and anatomical properties remains poorly understood. Here, we develop a hybrid modeling approach to bridge this gap. Our approach combines statistical and mechanistic models to predict cells electrophysiological activity from their gene expression pattern. To this end, we fit biophysical Hodgkin-Huxley-based models for a wide variety of cortical cell types using simulation-based inference, while overcoming the challenge posed by the mismatch between the mathematical model and the data. Using multimodal Patch-seq data, we link the estimated model parameters to gene expression using an interpretable sparse linear regression model. Our approach recovers specific ion channel gene expressions as predictive of biophysical model parameters including ion channel densities, directly implicating their mechanistic role in determining neural firing.

neuroscience↗

bioRxiv2022

Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks

Responses to natural stimuli in area V4 - a mid-level area of the visual ventral stream - are well predicted by features from convolutional neural networks (CNNs) trained on image classification. This result has been taken as evidence for the functional role of V4 in object classification. However, we currently do not know if and to what extent V4 plays a role in solving other computational objectives. Here, we investigated normative accounts of V4 (and V1 for comparison) by predicting macaque single-neuron responses to natural images from the representations extracted by 23 CNNs trained on different computer vision tasks including semantic, geometric, 2D, and 3D types of tasks. We found that V4 was best predicted by semantic classification features and exhibited high task selectivity, while the choice of task was less consequential to V1 performance. Consistent with traditional characterizations of V4 function that show its high-dimensional tuning to various 2D and 3D stimulus directions, we found that diverse non-semantic tasks explained aspects of V4 function beyond those captured by individual semantic tasks. Nevertheless, jointly considering the features of a pair of semantic classification tasks was sufficient to yield one of our top V4 models, solidifying V4s main functional role in semantic processing and suggesting that V4s affinity to 2D or 3D stimulus properties found by electrophysiologists can result from semantic functional goals.

neuroscience↗

bioRxiv2021

A flow-based latent state generative model of neural population responses to natural images

We present a joint deep neural system identification model for two major sources of neural variability: stimulus-driven and stimulus-conditioned fluctuations. To this end, we combine (1) state-of-the-art deep networks for stimulus-driven activity and (2) a flexible, normalizing flow-based generative model to capture the stimulus-conditioned variability including noise correlations. This allows us to train the model end-to-end without the need for sophisticated probabilistic approximations associated with many latent state models for stimulus-conditioned fluctuations. We train the model on the responses of thousands of neurons from multiple areas of the mouse visual cortex to natural images. We show that our model outperforms previous state-of-the-art models in predicting the distribution of neural population responses to novel stimuli, including shared stimulus-conditioned variability. Furthermore, it successfully learns known latent factors of the population responses that are related to behavioral variables such as pupil dilation, and other factors that vary systematically with brain area or retinotopic location. Overall, our model accurately accounts for two critical sources of neural variability while avoiding several complexities associated with many existing latent state models. It thus provides a useful tool for uncovering the interplay between different factors that contribute to variability in neural activity.

neuroscience↗

bioRxiv2021

Petascale neural circuit reconstruction: automated methods

3D electron microscopy (EM) has been successful at mapping invertebrate nervous systems, but the approach has been limited to small chunks of mammalian brains. To scale up to larger volumes, we have built a computational pipeline for processing petascale image datasets acquired by serial section EM, a popular form of 3D EM. The pipeline employs convolutional nets to compute the nonsmooth transformations required to align images of serial sections containing numerous cracks and folds, detect neuronal boundaries, label voxels as axon, dendrite, soma, and other semantic categories, and detect synapses and assign them to presynaptic and postsynaptic segments. The output of neuronal boundary detection is segmented by mean affinity agglomeration with semantic and size constraints. Pipeline operations are implemented by leveraging distributed and cloud computing. Intermediate results of the pipeline are held in cloud storage, and can be effortlessly viewed as images, which aids debugging. We applied the pipeline to create an automated reconstruction of an EM image volume spanning four visual cortical areas of a mouse brain. Code for the pipeline is publicly available, as is the reconstructed volume.

neuroscience↗

bioRxiv2021

Functional connectomics spanning multiple areas of mouse visual cortex

To understand the brain we must relate neurons functional responses to the circuit architecture that shapes them. Here, we present a large functional connectomics dataset with dense calcium imaging of a millimeter scale volume. We recorded activity from approximately 75,000 neurons in primary visual cortex (VISp) and three higher visual areas (VISrl, VISal and VISlm) in an awake mouse viewing natural movies and synthetic stimuli. The functional data were co-registered with a volumetric electron microscopy (EM) reconstruction containing more than 200,000 cells and 0.5 billion synapses. Subsequent proofreading of a subset of neurons in this volume yielded reconstructions that include complete dendritic trees as well the local and inter-areal axonal projections that map up to thousands of cell-to-cell connections per neuron. Here, we release this dataset as an open-access resource to the scientific community including a set of tools that facilitate data retrieval and downstream analysis. In accompanying papers we describe our findings using the dataset to provide a comprehensive structural characterization of cortical cell types1-3 and the most detailed synaptic level connectivity diagram of a cortical column to date2, uncovering unique cell-type specific inhibitory motifs that can be linked to gene expression data4. Functionally, we identify new computational principles of how information is integrated across visual space5, characterize novel types of neuronal invariances6 and bring structure and function together to decipher a general principle that wires excitatory neurons within and across areas7, 8.

neuroscience↗

bioRxiv2021

A CRISPR toolbox for generating intersectional genetic mice for functional, molecular, and anatomical circuit mapping

BackgroundA full understanding of circuits and cellular mechanisms governing health and disease requires the dissection and multi-faceted study of discrete cell subtypes in developing and adult animal models. Recombinase-driven expression of transgenic response alleles represents a significant and powerful approach to delineate cell populations for functional, molecular, and anatomical study. In addition to single recombinase systems, the expression of two recombinases in distinct, but partially overlapping, populations allow for more defined target expression. Although the application of this method is becoming increasingly popular, the expense and difficulty associated with production of customized intersectional mouse lines have limited widespread application to more common allele manipulations that are often commercially produced at great expense. ResultsWe present a simplified CRISPR toolkit for rapid, inexpensive, and facile intersectional allele production. Briefly, we produced 7 intersectional mouse lines using a dual recombinase system, one mouse line with a single recombinase system, and three embryonic stem (ES) cell lines that are designed to study how functional, molecular, and anatomical features relate to each other in building circuits that underlie physiology and behavior. As a proof-of-principle, we applied three of these lines to different neuronal populations for anatomical mapping and functional in vivo investigation of respiratory control. We also generated a mouse line with a single recombinase-responsive allele that controls the expression of the calcium sensor Twitch-2B. This mouse line was applied globally to study the effects of follicle stimulating hormone (FSH) and luteinizing hormone (LH) on calcium release in the ovarian follicle. ConclusionsLines presented here are representative examples of outcomes possible with the successful application of our genetic toolkit for the facile development of diverse, modifiable animal models. This toolkit will allow labs to create single or dual recombinase effector lines easily for any cell population or subpopulation of interest when paired with the appropriate Cre and FLP recombinase mouse lines or viral vectors. We have made our tools and derivative intersectional mouse and ES cell lines openly available for non-commercial use through publicly curated repositories for plasmid DNA, ES cells, and transgenic mouse lines.

genetics↗

bioRxiv2020

Multiscale and multimodal reconstruction of cortical structure and function

We present a semi-automated reconstruction of L2/3 mouse primary visual cortex from 3 million cubic microns of electron microscopic images, including pyramidal and inhibitory neurons, astrocytes, microglia, oligodendrocytes and precursors, pericytes, vasculature, mitochondria, and synapses. Visual responses of a subset of pyramidal cells are included. The data are being made publicly available, along with tools for programmatic and 3D interactive access. The density of synaptic inputs onto inhibitory neurons varies across cell classes and compartments. We uncover a compartment-specific correlation between mitochondrial coverage and synapse density. Frequencies of connectivity motifs in the graph of pyramidal cells are predicted quite accurately from node degrees using the configuration model of random graphs. Cells receiving more connections from nearby cells exhibit stronger and more reliable visual responses. These example findings illustrate the resources utility for relating structure and function of cortical circuits as well as for neuronal cell biology.

neuroscience↗

bioRxiv2020

Object manifold geometry across the mouse cortical visual hierarchy

Despite variations in appearance, we recognize objects robustly. Neuronal populations responding to objects presented under varying transformations form object manifolds, and hierarchically organized visual areas are hypothesized to untangle pixel intensities into linearly decodable object representations. However, the associated changes in the geometry of the object manifolds along the cortex remain unknown. Using home cage training, we showed that mice are capable of invariant object recognition. We recorded the responses of thousands of neurons to measure the information about object identity across the visual cortex and found that the lateral areas LM, LI, and AL carry more linearly decodable object information compared to other visual areas. We applied the theory of linear separability of manifolds and found that the increase in classification capacity is associated with a decrease in the dimension and radius of the object manifold, identifying the key features in the geometry of the population neural code that are associated with invariant object coding.

neuroscience↗

bioRxiv2020

Chandelier cell anatomy and function reveal a variably distributed but common signal

The activity and connectivity of inhibitory cells has a profound impact on the operation of neuronal networks. While the average connectivity of many inhibitory cell types has been characterized, we still lack an understanding of how individual interneurons distribute their synapses onto their targets and how heterogeneous the inhibition is onto different individual excitatory neurons. Here, we use large-scale volumetric electron microscopy (EM) and functional imaging to address this question for chandelier cells in layer 2/3 of mouse visual cortex. Using dense morphological reconstructions from EM, we mapped the complete chandelier input onto 153 pyramidal neurons. We find that the number of input synapses is highly variable across the population, but the variability is correlated with structural features of the target neuron: soma depth, soma size, and the number of perisomatic synapses received. Functionally, we found that chandelier cell activity in vivo was highly correlated and tracks pupil diameter, a proxy for arousal state. We propose that chandelier cells provide a global signal whose strength is individually adjusted for each target neuron. This approach, combining comprehensive structural analysis with functional recordings of identified cell types, will be a powerful tool to uncover the wiring rules across the diversity of cortical cell types.

neuroscience↗