Physics-Informed Generative Model for 3D Localization Microscopy

Abstract

Localization microscopy has overcome the diffraction limit, i.e. the conventional resolution limit of a microscope, enabling nanoscale biological imaging by precisely determining the positions of individual emitters such as single fluorescent molecules. However, the performance of deep learning methods, commonly applied to these tasks, depends significantly on the quality of training data, typically generated through simulation. Creating simulations that perfectly replicate experimental conditions remains challenging, resulting in a persistent simulation-to-experiment gap. To bridge this gap, we propose a physics-informed generative model leveraging self-supervised learning directly on experimental data. Our model extends the Deep Latent Particles (DLP) framework by incorporating a physical model of the Point Spread Function (PSF; the image of a single point source in the microscope) into the decoder, enabling it to disentangle learned realistic environments from emitters. Trained directly on unlabeled experimental images, our model intrinsically captures realistic background, noise patterns, and emitter characteristics. The decoder thus acts as a high-fidelity generator, producing fully labeled, realistic training images with known emitter locations. Using these generated datasets significantly improves the performance of supervised localization networks, particularly in challenging scenarios such as complex backgrounds and low signal-to-noise ratios. We demonstrate our approach on a variety of experimentally measured microscopy data, including super-resolution imaging in 2D and 3D and particle tracking in live cells, showing substantial improvements in localization precision and emitter detection.