OmniT2I: a unified table-to-image framework for survival prediction and cross-modal inference with incomplete multimodal cancer data
Molecular profiles and histopathology provide complementary information about cancer prognosis, but these data are often incomplete for individual patients. Here we present OmniT2I, a table-to-image framework for survival prediction and cross-modal inference from incomplete multimodal cancer data. Each molecular profile was converted into a three-channel image representation with correlation-based feature placement. OmniT2I combines available molecular and histopathology embeddings through availability-aware fusion and predicts missing modalities from those observed. Across 10,373 patients with 32 cancer types, the C-index was 0.631 within cancer types and 0.783 with all cancers pooled (0.660 and 0.787 in 5,605 patients with all five modalities). A single model handled all 31 modality combinations and, compared with nine survival methods trained separately for each, achieved higher C-indices in 240 of 279 pooled and 226 of 279 within-cancer comparisons. Adding predictions of withheld modalities to observed inputs improved pooled C-indices in all 180 pairings of observed and withheld modality sets and within-cancer C-indices in 164. Adding measured histopathology to RNA produced the largest within-cancer C-index gain among the four additional modalities evaluated, whereas adding copy number variation reduced it. In seven external cohorts, frozen RNA-only C-indices differed from test performance in the corresponding cancer type in TCGA by a median of -0.003 (range, -0.092 to +0.079). OmniT2I enables prognostic modeling with incomplete multimodal data.