Biology subjects

Pfister, H.

Publications and source records attributed to Pfister, H..

At least 19 recordsLinked to original sources

bioRxiv2026

RnR-ExM: Robust Non-rigid Registration Challenge for Expansion Microscopy Volumes

Expansion Microscopy (ExM) enables nanoscale imaging of biological samples while preserving structural and molecular details. The Robust Non-rigid Registration Challenge for ExM (RnR-ExM) was launched to benchmark 3D registration algorithms on real-world ExM datasets. The challenge released 24 pairs of high-resolution 3D volumes from zebrafish, mouse, and C.~elegans, deliberately selected for their difficulty with standard alignment methods. To evaluate performance, we provided curated segmentation masks of nuclei and blood vessels as landmarks for half of the dataset. To encourage learning-based approaches, we released code for generating synthetic training data reflecting the deformation and appearance of the unlabeled pairs. By submission, 168 participants registered, with six teams submitting results by the deadline; the top three were invited to present at ISBI 2023. Accurate registration of ExM volumes is essential for downstream biological analyses such as multiplexed molecular mapping and cell-type-specific structural profiling. In this paper, we detail the methods and results of these top teams, highlighting current capabilities and future directions for 3D registration in ExM. All datasets and code are available at https://rnr-exm.grand-challenge.org/

neuroscience↗

bioRxiv2026

Clocks and Dominoes: Timing Mechanisms of Embryogenesis

How developmental timings are regulated is a fundamental open question. Two widely considered mechanisms are the clock, in which an internal timer determines when each stage occurs, and the domino, in which the completion of each stage triggers the next. It is often unclear how to establish either mechanism. Here, we construct a quantitative framework that uses the correlation structure of developmental timings to test the clock and domino mechanisms. We apply this framework to human pre-implantation development by using ~1 million images of 2946 embryos acquired during IVF treatment, establishing mathematical models of developmental rate. We find that a domino mechanism governs the cleavage timings, while a pronuclei fade-triggered clock mechanism governs the morula and blastocyst timings. These results are consistent with the cell cycle oscillator governing the cleavage timings and the accumulation of embryonic gene products or the degradation of maternally deposited factors governing the morula and blastocyst timings. We next investigate the physiological regulators of developmental timing by analyzing how the timings are statistically associated with the clinical pregnancy outcome. While embryos that result in a clinical pregnancy tend to exhibit shorter cleavage timings, this association is primarily driven by patient-specific properties. In contrast, embryo-specific properties independently influence the pregnancy outcome and the cleavage timings, so that factors directly determining implantation potential, such as aneuploidy, can only weakly impact the cleavage timings. Taken together, this work provides a robust framework for decoding developmental timing mechanisms, with significant implications for fundamental biology and clinical practice.

developmental biology↗

bioRxiv2025

Correcting Non-Uniform Milling in FIB-SEM Images with Unsupervised Cross-Plane Image-to-Image Translation

MotivationFocused Ion Beam Scanning Electron Microscopy (FIB-SEM) is an advanced Volume Electron Microscopy technology with growing applications, featuring thinner sectioning compared to other Volume Electron Microscopes. Such axial resolution is crucial for accurate segmentation and reconstruction of fine structures in biological tissues. However, in reality, the milling thickness is not always uniform across the sample surface, resulting in the axial plane looking distorted. Existing image processing approaches often: (i) assume constant section thickness; (ii) consist of multiple separate processing steps (i.e., not in an end-to-end fashion); (iii) require ground truth images for modeling, which may entail significant labor and be unsuitable for rapid analysis. ResultsWe develop a deep learning method to correct non-uniform milling artifacts observed in FIB-SEM images. The proposed method is an image-to-image translation technique that can mitigate image distortions in an unsupervised manner. It conducts cross-plane learning within 3D image volumes without any ground truth annotations. We demonstrate the efficacy of our method on a real-world micro-wasp dataset, showcasing significantly improved image quality after correction with qualitative and quantitative analysis. Contactpfister@seas.harvard.edu, jingpengw@lglab.ac.cn

bioinformatics↗

bioRxiv2025

Developmental Connectomics of the Mouse Cerebellum

To uncover the developmental processes that establish the precise patterns of synaptic connectivity in the CNS, we employed a connectomic approach in the mouse cerebellar cortex between birth and 2 weeks of age. There were dramatic quantitative and qualitative changes in the structure and connectivity of cerebellar cells. Parallel fiber synapses onto Purkinje cells increased [~]500-fold, with the most rapid growth taking place a week after birth. To support this profound synaptogenesis, Purkinje cells generated thousands of transient parallel fiber-oriented filopodia that received nascent synapses from parallel fibers. Importantly, we find that granule cells initiate synaptic output onto Purkinje cells only after receiving mossy fiber input, revealing a sequential, input-dependent logic for circuit assembly. In sharp contrast to the concurrent pruning of climbing fiber inputs, parallel fiber connectivity expanded and became highly individualized during development. Despite anatomical overlap, neighboring Purkinje cells share significantly fewer parallel fiber inputs than expected by chance. Moreover, parallel fibers themselves diverged spatially, further enforcing selective input allocation and resulting in highly specific parallel fiber cohorts for each Purkinje cell. Our findings uncover a mechanistic sequence in which early afferent activity and transient cellular structures guide the selective wiring and expansion of parallel fiber input to Purkinje cells, establishing developmental principles that ensure functional specificity in the mature brain.

neuroscience↗

bioRxiv2025

SynAnno: Interactive Guided Proofreading of Synaptic Annotations

Connectomics, a subfield of neuroscience, aims to map and analyze synapse-level wiring diagrams of the nervous system. While recent advances in deep learning have accelerated automated neuron and synapse segmentation, reconstructing accurate connectomes still demands extensive human proofreading to correct segmentation errors. We present SynAnno, an interactive tool designed to streamline and enhance the proofreading of synaptic annotations in large-scale connectomics datasets. SynAnno integrates into existing neuroscience workflows by enabling guided, neuron-centric proofreading. To address the challenges posed by the complex spatial branching of neurons, it introduces a structured workflow with an optimized traversal path and a 3D mini-map for tracking progress. In addition, SynAnno incorporates fine-tuned machine learning models to assist with error detection and correction, reducing the manual burden and increasing proofreading efficiency. We evaluate SynAnno through a user and case study involving seven neuroscience experts. Results show that SynAnno significantly accelerates synapse proofreading while reducing cognitive load and annotation errors through structured guidance and visualization support. The source code and interactive demo are available at: https://github.com/PytorchConnectomics/SynAnno.

neuroscience↗

bioRxiv2025

SEAL: Spatially-resolved Embedding Analysis with Linked Imaging Data

Dimensionality reduction techniques help analysts make sense of complex, high-dimensional spatial datasets, such as multiplexed tissue imaging, satellite imagery, and astronomical observations, by projecting data attributes into a two-dimensional space. However, these techniques typically abstract away crucial spatial, positional, and morphological contexts, complicating interpretation and limiting insights. To address these limitations, we present SEAL, an interactive visual analytics system designed to bridge the gap between abstract 2D embeddings and their rich spatial imaging context. SEAL introduces a novel hybrid-embedding visualization that preserves image and morphological information while integrating critical high-dimensional feature data. By adapting set visualization methods, SEAL allows analysts to identify, visualize, and compare selections--defined manually or algorithmically--in both the embedding and original spatial views, facilitating a deeper understanding of the spatial arrangement and morphological characteristics of entities of interest. To elucidate differences between selected sets of items, SEAL employs a scalable surrogate model to calculate feature importance scores, identifying the most influential features governing the position of objects within embeddings. These importance scores are visually summarized across selections, with mathematical set operations enabling detailed comparative analyses. We demonstrate SEALs effectiveness and versatility through three case studies: colorectal cancer tissue analysis with a pharmacologist, melanoma investigation with a cell biologist, and exploration of sky survey data with an astronomer. These studies underscore the importance of integrating image context into embedding spaces when interpreting complex imaging datasets. Implemented as a standalone tool while also integrating seamlessly with computational notebooks, SEAL provides an interactive platform for spatially informed exploration of high-dimensional datasets, significantly enhancing interpretability and insight generation.

bioinformatics↗

bioRxiv2025

MoMo - Combining Neuron Morphology and Connectivity forInteractive Motif Analysis in Connectomes

Connectomics, a subfield of neuroscience, reconstructs structural and functional brain maps at synapse-level resolution. These complex spatial maps consist of tree-like neurons interconnected by synapses. Motif analysis is a widely used method for identifying recurring subgraph patterns in connectomes. These motifs, thus, potentially represent fundamental units of information processing. However, existing computational tools often oversimplify neurons as mere nodes in a graph, disregarding their intricate morphologies. In this paper, we introduce MoMo, a novel interactive visualization framework for analyzing neuron morphology-aware motifs in large connectome graphs. First, we propose an advanced graph data structure that integrates both neuronal morphology and synaptic connectivity. This enables highly efficient, parallel subgraph isomorphism searches, allowing for interactive morphological motif queries. Second, we develop a sketch-based interface that facilitates the intuitive exploration of morphology-based motifs within our new data structure. Users can conduct interactive motif searches on state-of-the-art connectomes and visualize results as interactive 3D renderings. We present a detailed goal and task analysis for motif exploration in connectomes, incorporating neuron morphology. Finally, we evaluate MoMo through case studies with four domain experts, who asses the tools usefulness and effectiveness in motif exploration, and relevance to real-world neuroscience research. The source code for MoMo is available here.

neuroscience↗

bioRxiv2025

Morphology-Aware Profiling of Highly Multiplexed Tissue Images using Variational Autoencoders

Spatial proteomics (highly multiplexed tissue imaging) provides unprecedented insight into the types, states, and spatial organization of cells within preserved tissue environments. To enable single-cell analysis, high-plex images are typically segmented using algorithms that assign marker signals to individual cells. However, conventional segmentation is often imprecise and susceptible to signal spillover between adjacent cells, interfering with accurate cell type identification. Segmentation-based methods also fail to capture the morphological detail that histopathologists rely on for disease diagnosis and staging. Here, we present a method that combines unsupervised, pixel-level machine learning using autoencoders with traditional segmentation to generate single-cell data that captures information on protein abundance, morphology, and local neighborhood in a manner analogous to human experts while overcoming signal spillover. We demonstrate the generality of this approach by applying it to CyCIF, Lunaphore COMET, and CODEX data, and show that it can learn histological features across multiple spatial scales.

pathology↗

bioRxiv2024

Global Neuron Shape Reasoning with Point Affinity Transformers

Connectomics is a field of neuroscience that maps the brains intricate wiring diagram. Accurate neuron segmentation from microscopy volumes is essential for automating connectome reconstruction. However, state-of-the-art algorithms use image-based convolutional neural networks limited to local neuron shape context. Thus, we introduce a new framework that reasons over global neuron shape with a novel point affinity transformer. Our framework embeds a (multi-)neuron point cloud into a fixed-length feature set from which we can decode any point pair affinities, enabling clustering neuron point clouds for automatic proofreading. We also show that the learned feature set can easily be mapped to a contrastive embedding space that enables neuron type classification using a simple classifier. Our approach excels in two demanding connectomics tasks: correcting segmentation errors and classifying neuron types. Evaluated on three benchmark datasets derived from state-of-the-art connectomes, our method outperforms point transformers, graph neural networks, and unsupervised clustering baselines.

neuroscience↗

bioRxiv2024

niiv: Fast Self-supervised Neural ImplicitIsotropic Volume Reconstruction

Three-dimensional (3D) microscopy data often is anisotropic with significantly lower resolution (up to 8x) along the z axis than along the xy axes. Computationally generating plausible isotropic resolution from anisotropic imaging data would benefit the visual analysis of large-scale volumes. This paper proposes niiv, a self-supervised method for isotropic reconstruction of 3D microscopy data that can quickly produce images at arbitrary output resolutions. The representation embeds a learned latent code within a neural field that describes the implicit higher-resolution isotropic image region. We use a novel attention-guided latent interpolation approach, which allows flexible information exchange over a local latent neighborhood. Under isotropic volume assumptions, we self-supervise this representation on low-/high-resolution lateral image pairs to reconstruct an isotropic volume from low-resolution axial images. We evaluate our method on simulated and real anisotropic electron (EM) and light microscopy (LM) data. Compared to a state-of-the- art diffusion-based method, niiv shows improved reconstruction quality (+1 dB PSNR) and is over three orders of magnitude faster (1,000x) to infer. Specifically, niiv reconstructs a 1283 voxel volume in 2/10th of a second, renderable at varying (continuous) high resolutions for display.

bioinformatics↗

bioRxiv2024

EM-Compressor: Electron Microscopy Image Compression in Connectomics with Variational Autoencoders

The ongoing pursuit to map detailed brain structures at high resolution using electron microscopy (EM) has led to advancements in imaging that enable the generation of connectomic volumes that have reached the petabyte scale and are soon expected to reach the exascale for whole mouse brain collections. To tackle the high costs of managing these large-scale datasets, we have developed a data compression approach employing Variational Autoencoders (VAEs) to significantly reduce data storage requirements. Due to their ability to capture the complex patterns of EM images, our VAE models notably decrease data size while carefully preserving important image features pertinent to connectomics-based image analysis. Through a comprehensive study using human EM volumes (H01 dataset), we demonstrate how our approach can reduce data to as little as 1/128th of the original size without significantly compromising the ability to subsequently segment the data, outperforming standard data size reduction methods. This performance suggests that this method can greatly alleviate requirements for data management for connectomics applications, and enable more efficient data access and sharing. Additionally, we developed a cloud-based application named EM-Compressor on top of this work to enable on-thefly interactive visualization: https://em-compressor-demonstration.s3.amazonaws.com/EM-Compressor+App.mp4.

neuroscience↗

bioRxiv2024

Anatomics MLT, an AI tool for large scale quantification of ultrastructural traits

The ever increasing breadth of biological knowledge has led to recent efforts to combine information from various fields into cell- or tissue atlases. Anatomical features are the structural basis for such efforts, but unfortunately large scale analysis of subcellular anatomical traits is currently a missing feature. Similarly, small phenotypic alterations of organelle- or cell-specific anatomical traits, such as an increase of the total volume or the number of mitochondria in response to certain stimuli, are currently hard to quantify. To provide tools to extract quantitative information from available 3D microscopic datasets generated with methods such as serial block face scanning electron microscopy we a) developed much improved fixation and embedding protocols for plants to drastically reduce processing artifacts and b) generated an easy-to-use AI tool for quantitative analysis and visualization of large-scale data sets. We make this tool available as open source.

cell biology↗

bioRxiv2024

psudo: Exploring Multi-Channel Biomedical Image Data with Spatially and Perceptually Optimized Pseudocoloring

Over the past century, multichannel fluorescence imaging has been pivotal in myriad scientific breakthroughs by enabling the spatial visualization of proteins within a biological sample. With the shift to digital methods and visualization software, experts can now flexibly pseudocolor and combine image channels, each corresponding to a different protein, to explore their spatial relationships. We thus propose psudo, an interactive system that allows users to create optimal color palettes for multichannel spatial data. In psudo, a novel optimization method generates palettes that maximize the perceptual differences between channels while mitigating confusing color blending in overlapping channels. We integrate this method into a system that allows users to explore multi-channel image data and compare and evaluate color palettes for their data. An interactive lensing approach provides on-demand feedback on channel overlap and a color confusion metric while giving context to the underlying channel values. Color palettes can be applied globally or, using the lens, to local regions of interest. We evaluate our palette optimization approach using three graphical perception tasks in a crowdsourced user study with 150 participants, showing that users are more accurate at discerning and comparing the underlying data using our approach. Additionally, we showcase psudo in a case study exploring the complex immune responses in cancer tissue data with a biologist. CCS ConceptsO_LIHuman-centered computing [->] Visualization systems and tools; C_LI

cell biology↗

bioRxiv2023

A large-scale volumetric correlated light and electron microscopy study localizes Alzheimer's disease-related molecules in the hippocampus

Connectomics using volume-electron-microscopy enables mapping and analysis of neuronal networks, revealing insights into neural circuit function and dysfunction. In Alzheimers disease (AD), where amyloid-{beta} (A{beta}) and hyperphosphorylated-Tau (pTau) are implicated, connectomics offers an approach to unravel how these molecules contribute to circuit alterations by enabling the study of these molecules within the context of the complete local neuronal and glial milieu. We present a volumetric-correlated-light-and-electron microscopy (vCLEM) protocol using fluorescent nanobodies to localize A{beta} and pTau within a large-scale connectomics dataset from the hippocampus of the 3xTg AD mouse model. A key outcome of this work is a publicly accessible vCLEM dataset, featuring fluorescent labeling of A{beta} and pTau in the ultrastructural context with segmented neurons, glia, and synapses. This dataset provides a unique resource for exploring AD pathology in the context of connectomics and fosters collaborative opportunities in neurodegenerative disease research. As a proof-of-principle, we uncovered new localizations of A{beta} and pTau, including pTau-positive spine-like protrusions at the axon initial segment and changes in the number and size of synapses near A{beta} plaques. Our vCLEM approach facilitates the discovery of both molecular and structural alterations within large-scale EM data, advancing connectomics research in Alzheimers and other neurodegenerative diseases.

neuroscience↗

bioRxiv2023

SmartEM: machine-learning guided electron microscopy

Connectomics provides nanometer-resolution, synapse-level maps of neural circuits to understand brain activity and behavior. However, few researchers have access to the high-throughput electron microscopes necessary to generate enough data for whole brain or even whole circuit reconstruction. To date, machine-learning methods have been used after the collection of images by electron microscopy (EM) to accelerate and improve neuronal segmentation, synapse reconstruction and other data analysis. With the computational improvements in processing EM images, acquiring EM images has now become the rate-limiting step in automated connectomics. Here, in order to speed up EM imaging, we integrate machine-learning into real-time image acquisition in a single-beam scanning electron microscope. This SmartEM approach allows an electron microscope to perform intelligent, data-aware imaging of specimens. SmartEM saves time by allocating the proper imaging time for each region of interest - first scanning all pixels rapidly, then rescanning more slowly only the small subareas where a higher quality signal is required. We demonstrate that SmartEM achieves up to a ~7-fold acceleration of image acquisition time for connectomic samples using a commercial single-beam SEM in samples from nematodes, mice and human brain. We apply this fast imaging method to reconstruct a portion of mouse cerebral cortex with an accuracy comparable to traditional electron microscopy.

neuroscience↗

bioRxiv2023

CAVE: Connectome Annotation Versioning Engine

Advances in Electron Microscopy, image segmentation and computational infrastructure have given rise to large-scale and richly annotated connectomic datasets which are increasingly shared across communities. To enable collaboration, users need to be able to concurrently create new annotations and correct errors in the automated segmentation by proofreading. In large datasets, every proofreading edit relabels cell identities of millions of voxels and thousands of annotations like synapses. For analysis, users require immediate and reproducible access to this constantly changing and expanding data landscape. Here, we present the Connectome Annotation Versioning Engine (CAVE), a computational infrastructure for immediate and reproducible connectome analysis in up-to petascale datasets ([~]1mm3) while proofreading and annotating is ongoing. For segmentation, CAVE provides a distributed proofreading infrastructure for continuous versioning of large reconstructions. Annotations in CAVE are defined by locations such that they can be quickly assigned to the underlying segment which enables fast analysis queries of CAVEs data for arbitrary time points. CAVE supports schematized, extensible annotations, so that researchers can readily design novel annotation types. CAVE is already used for many connectomics datasets, including the largest datasets available to date.

neuroscience↗

bioRxiv2023

Multiplexed Volumetric CLEM enabled by antibody derivatives provides new insights into the cytology of the mouse cerebellar cortex

Mapping neuronal networks that underlie behavior has become a central focus in neuroscience. While serial section electron microscopy (ssEM) can reveal the fine structure of neuronal networks (connectomics), it does not provide the molecular information that helps identify cell types or their functional properties. Volumetric correlated light and electron microscopy (vCLEM) combines ssEM and volumetric fluorescence microscopy to incorporate molecular labeling into ssEM datasets. We developed an approach that uses small fluorescent single-chain variable fragment (scFv) immuno-probes to perform multiplexed detergent-free immuno-labeling and ssEM on the same samples. We generated eight such fluorescent scFvs that targeted useful markers for brain studies (green fluorescent protein, glial fibrillary acidic protein, calbindin, parvalbumin, voltage-gated potassium channel subfamily A member 2, vesicular glutamate transporter 1, postsynaptic density protein 95, and neuropeptide Y). To test the vCLEM approach, six different fluorescent probes were imaged in a sample of the cortex of a cerebellar lobule (Crus 1), using confocal microscopy with spectral unmixing, followed by ssEM imaging of the same sample. The results show excellent ultrastructure with superimposition of the multiple fluorescence channels. Using this approach we could document a poorly described cell type in the cerebellum, two types of mossy fiber terminals, and the subcellular localization of one type of ion channel. Because scFvs can be derived from existing monoclonal antibodies, hundreds of such probes can be generated to enable molecular overlays for connectomic studies.

neuroscience↗

bioRxiv2022

Vimo: Visual Analysis of Neuronal Connectivity Motifs

Recent advances in high-resolution connectomics provide researchers with access to accurate petascale reconstructions of neuronal circuits and brain networks for the first time. Neuroscientists are analyzing these networks to better understand information processing in the brain. In particular, scientists are interested in identifying specific small network motifs, i.e., repeating subgraphs of the larger brain network that are believed to be neuronal building blocks. Although such motifs are typically small (e.g., 2 - 6 neurons), the vast data sizes and intricate data complexity present significant challenges to the search and analysis process. To analyze these motifs, it is crucial to review instances of a motif in the brain network and then map the graph structure to detailed 3D reconstructions of the involved neurons and synapses. We present Vimo, an interactive visual approach to analyze neuronal motifs and motif chains in large brain networks. Experts can sketch network motifs intuitively in a visual interface and specify structural properties of the involved neurons and synapses to query large connectomics datasets. Motif instances (MIs) can be explored in high-resolution 3D renderings. To simplify the analysis of MIs, we designed a continuous focus&context metaphor inspired by visual abstractions. This allows users to transition from a highly-detailed rendering of the anatomical structure to views that emphasize the underlying motif structure and synaptic connectivity. Furthermore, Vimo supports the identification of motif chains where a motif is used repeatedly (e.g., 2 - 4 times) to form a larger network structure. We evaluate Vimo in a user study and an in-depth case study with seven domain experts on motifs in a large connectome of the fruit fly, including more than 21,000 neurons and 20 million synapses. We find that Vimo enables hypothesis generation and confirmation through fast analysis iterations and connectivity highlighting.

neuroscience↗