bioRxiv · 10.1101/2021.07.19.452964
Task-Assisted GAN for Resolution Enhancement and Modality Translation in Fluorescence Microscopy
Abstract
AbstractWe introduce a deep learning model that predicts super-resolved versions of diffraction-limited microscopy images. Our model, named Task- Assisted Generative Adversarial Network (TA-GAN), incorporates an auxiliary task (e.g. segmentation, localization) closely related to the observed biological nanostructures characterization. We evaluate how TA-GAN improves generative accuracy over unassisted methods using images acquired with different modalities such as confocal, brightfield (diffraction-limited), super-resolved stimulated emission depletion, and structured illumination microscopy. The generated synthetic resolution enhanced images show an accurate distribution of the F-actin nanostructures, replicate the nanoscale synaptic cluster morphology, allow to identify dividing S. aureus bacterial cell boundaries, and localize nanodomains in simulated images of dendritic spines. We expand the applicability of the TA-GAN to different modalities, auxiliary tasks, and online imaging assistance. Incorporated directly into the acquisition pipeline of the microscope, the TA-GAN informs the user on the nanometric content of the field of view without requiring the acquisition of a super-resolved image. This information is used to optimize the acquisition sequence, and reduce light exposure. The TA-GAN also enables the creation of domain-adapted labeled datasets requiring minimal manual annotation, and assists microscopy users by taking online decisions regarding the choice of imaging modality and regions of interest.
Source connections
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Bouchard, C., Wiesner, T., Deschenes, A., Lavoie-Cardinal, F., Gagne, C.. 2021-07-20. Task-Assisted GAN for Resolution Enhancement and Modality Translation in Fluorescence Microscopy. https://doi.org/10.1101/2021.07.19.452964
Cite the original work for its findings. Save a collection to share your selection of sources.