Vision Transformers as a Framework for Imaging Fluorescence Correlation Spectroscopy

Abstract

Fluorescence correlation spectroscopy (FCS) is widely used for quantifying molecular dynamics in live cells. However, conventional FCS relies on the autocorrelation function (ACF), a lossy transformation that discards higher-order information, mathematically couples distinct biophysical parameters, and requires long measurements (~50,000 frames, or ~1 minute) that obscure short-lived transient events. To overcome this information bottleneck, we introduce ImFCSViT, a spatiotemporal Vision Transformer-based framework that bypasses the ACF representation entirely, operating directly on raw fluorescence image stacks. By introducing a dedicated normalization token to preserve global intensity statistics, our approach simultaneously extracts four biophysical parameters (diffusion coefficient D, particle density N, particle brightness B, and PSF structure factor {sigma}0) in a single forward pass. Requiring only 2.5 s of data, and potentially down to 1 s (a 1-2 order reduction in data requirement), our approach accurately recovers diffusion coefficients while utilizing its spatial field-of-view to reveal spatially heterogeneous, slow-diffusing sub-populations in lipid bilayers and live-cell plasma membranes that are invisible to classical FCS. By leveraging the data-driven representational learning capabilities of deep learning, we present a flexible, generalizable, and powerful tool for mapping complex, multi-parameter spatiotemporal dynamics in biological systems.