Fine-Grained and Reliable Cell Classification from Pathology Images

Abstract

Molecular assays such as spatial transcriptomics facilitate fine-grained identification of cell types for analyzing tumor microenvironments. However, they are expensive to acquire in routine clinical practice compared to hematoxylin and eosin (H&E) images. With molecular guidance, such images can distinguish cell lineages, yet often fail to resolve closely related subtypes. To overcome this, we introduce CellBridge, a multi-modal contrastive learning framework pretrained on 16M cells with paired H&E and spatial profiles, for transferring molecular information into morphological representations. CellBridge further incorporates a Cell-Ontology-aware conformal prediction module that assigns labels at different levels of specificity: leaf-level cell types when morphology alone provides sufficient evidence, and broader ancestor categories when distinguishing among more specific cell types would require molecular information. Across in-domain and out-of-domain benchmarks, CellBridge outperforms leading pathology foundation models in macro F1 by up to 16 percentage points, with the largest gains on the most fine-grained label spaces. It consistently achieves smaller prediction sets, while reducing class-wise coverage gaps. We envision CellBridge as a step toward extracting cellular information that currently requires costly molecular assays from routinely acquired pathology images, while explicitly identifying where such assays remain necessary. Our code is available at https://github.com/Lotfollahi-lab/cellbridge.