Search bioRxiv⌕ Search

Biology subjects

Kather, J. N.

Publications and source records attributed to Kather, J. N..

At least 19 recordsLinked to original sources

Transforming Histology into Virtual Multiplex Immunofluorescence to Decode Prognostic Spatial Immunity in Hepatocellular Carcinoma

The spatial organization of the tumor immune microenvironment (TIME) drives hepatocellular carcinoma (HCC) prognosis but remains unquantifiable on routine H&E slides. Here, we present HCCExplorer, a deep learning framework that translates H&E into virtual multiplex immunofluorescence (mIF) and uses multi-modal graph learning to decode spatial survival signals. Trained on 30 H&E-mIF slide pairs, HCCExplorer evaluated a 1,813-slide multi-center cohort. It achieved superior overall survival stratification over clinical indices and state-of-the-art pathology and protein foundation models, yielding a concordance index of 0.71 and a Hazard Ratio (HR) of 15.46 (P < 0.001), maintaining stability across three external cohorts. Beyond risk stratification, interpretation of model features identified M1 macrophage infiltration as a protective determinant (HR=0.40, P < 0.05). Furthermore, it uncovered a protective "Containment Niche" at the invasion frontier (HR=0.02, P < 0.01), featuring macrophages co-localizing with Foxp3+ Tregs and CD4+ T cells. Ultimately, HCCExplorer provides actionable, spatially-resolved biomarkers from conventional histology for precision HCC management.

pathology↗

Deep Learning Predicts Survival Across Squamous Tumor Entities From H&E Stains: Insights from Head and Neck, Esophagus, Lung and Cervical Cancer

Computational pathology-based models are becoming increasingly popular for extracting biomarkers from images of cancer tissue. However, their validity is often only demonstrated on a single unseen validation cohort, limiting insights into their generalizability and posing challenges for explainability. In this study, we developed models to predict overall survival using haematoxylin and eosin (H&E) slides from FFPE samples in head and neck squamous cell carcinoma (HNSCC). By validating our models across diverse squamous tumor entities, including, head and neck (HR = 1.58, 95% CI = 1.17-2.12, p = 0.003), esophageal (non-significant), lung (HR = 1.31, 95% CI = 1.13-1.52, p < 0.001) and cervical (HR = 1.39, 95% CI = 1.10-1.75, p = 0.005) squamous cell carcinomas, we showed that the predicted risk score captures relevant information for survival beyond HNSCC. Correlation analysis indicated that the predicted risk score is strongly associated with various clinical factors, including human papillomavirus status, tumor volume and smoking history, although the specific factors vary across cohorts. These results emphasize the necessity for comprehensive validation and in-depth assessment of computational pathology-based models to better characterize the underlying patterns they learn during training.

pathology↗

IBDome: An integrated molecular, histopathological, and clinical atlas of inflammatory bowel diseases

Multi-omic and multimodal datasets with detailed clinical annotations offer significant potential to advance our understanding of inflammatory bowel diseases (IBD), refine diagnostics, and enable personalized therapeutic strategies. In this multi-cohort study, we performed an extensive multi-omic and multimodal analysis of 1,002 clinically annotated patients with IBD and non-IBD controls, incorporating whole-exome and RNA sequencing of normal and inflamed gut tissues, serum proteomics, and histopathological assessments from images of H&E-stained tissue sections. Transcriptomic profiles of normal and inflamed tissues revealed distinct site-specific inflammatory signatures in Crohns disease (CD) and ulcerative colitis (UC). Leveraging serum proteomics, we developed an inflammatory protein severity signature that reflects underlying intestinal molecular inflammation. Furthermore, foundation model-based deep learning accurately predicted histologic disease activity scores from images of H&E-stained intestinal tissue sections, offering a robust tool for clinical evaluation. Our integrative analysis highlights the potential of combining multi-omics and advanced computational approaches to improve our understanding and management of IBD.

molecular biology↗

SwarmMAP: Swarm Learning for Decentralized Cell Type Annotation in Single Cell Sequencing Data

Rapid technological advancements have made it possible to generate single-cell data at a large scale. Several laboratories around the world can now generate single-cell transcriptomic data from different tissues. Unsupervised clustering, followed by annotation of the cell type of the identified clusters, is a crucial step in single-cell analyses. However, there is no consensus on the marker genes to use for annotation, and celltype annotation is currently mostly done by manual inspection of marker genes, which is irreproducible, and poorly scalable. Additionally, patient-privacy is also a critical issue with human datasets. There is a critical need to standardize and automate celltype annotation across datasets in a privacy-preserving manner. Here, we developed SwarmMAP that uses Swarm Learning to train machine learning models for cell-type classification based on single-cell sequencing data in a decentralized way. SwarmMAP does not require any exchange of raw data between data centers. SwarmMAP has a F1-score of 0.93, 0.98, and 0.88 for cell type classification in human heart, lung, and breast datasets, respectively. Swarm Learning-based models yield an average performance of 0.907 which is on par with the performance achieved by models trained on centralized data (p-val=0.937, Mann-Whitney U Test). We also find that increasing the number of datasets increases cell-type prediction accuracy and enables handling higher cell-type diversity. Together, these findings demonstrate that Swarm Learning is a viable approach to automate cell-type annotation. SwarmMAP is available at https://github.com/hayatlab/SwarmMAP.

bioinformatics↗

Deep Learning for Biomarker Discovery in Cancer Genomes

BackgroundAccurate determination of genomic biomarkers from tumor sequencing is fundamental to precision oncology, informing disease classification and treatment decisions. In practice, biomarker inference relies on computational pipelines that often compress high-dimensional mutation data into predefined summaries such as mutational signatures or composite genomic features. While robust and widely adopted, these representations may not fully capture the complexity of cancer genomes. Deep learning (DL) offers an end-to-end alternative by learning features directly from raw genomic data. However, clinical translation remains challenging due to limited empirical validation of new DL models and a lack of systematic comparisons with established machine learning (ML) baselines, particularly when transitioning from information-rich genome or exome data to real-world targeted sequencing profiles. Here, we compare state-of-the-art DL architectures with classical ML models across variant-level, copy-number (CNV), and multimodal inputs, using microsatellite instability (MSI) and homologous recombination deficiency (HRD) prediction as oncologically relevant tasks. We aim to derive practical guidance on modelling strategies across different data modalities and clinical sequencing contexts. MethodsFor MSI and HRD prediction, we trained multiple DL models, including supervised and self-supervised encoders, alongside feature-based ML approaches using tumor mutation data, copy-number alterations, and their multimodal combinations. Analyses were conducted on 5,647 patients in The Cancer Genome Atlas (TCGA), the Clinical Proteomic Tumor Analysis Consortium (CPTAC), and two targeted sequencing panel cohorts. Model performance was evaluated on both whole-exome and panel-based datasets, and explainability analysis were performed for both DL and ML models. ResultsFor MSI, DL demonstrated stronger generalization than ML on external validation data (F1 0.97 vs 0.76) and maintained comparatively high performance under pseudo-panels conditions, whereas ML performance dropped. In a real-world targeted panel cohort, DL again showed more robust generalization than ML, with performance partly affected by cross-assay variability. For HRD, incorporation of CNV data was the primary determinant of predictive performance. Once CNVs were included, DL and ML achieved similar accuracy on external datasets (F1 0.61 vs 0.58). In panel-based settings, DL retained an advantage over ML (F1 0.78 vs 0.62). Model interpretation analyses indicated that both DL and ML relied on mutation and chromosomal patterns consistent with established MSI and HRD biology. ConclusionOverall, predictive performance depended strongly on data availability and clinical sequencing context. When information-rich inputs were available, both DL and classical ML achieved robust biomarker prediction, with DL generally matching or exceeding ML performance. The most pronounced advantages of DL emerged in cross-assay evaluations and data-sparse settings, where generalization was more reliable. Notably, the best-performing DL models were lightweight and interpretable, supporting practical deployment. In clinical genomics workflows, such models may complement established pipelines by leveraging patient sequencing data to provide additional evidence for treatment-relevant biomarker assessment.

bioinformatics↗

Counterfactual Diffusion Models for Mechanistic Explainability of Artificial Intelligence Models in Pathology

Deep learning can extract predictive and prognostic biomarkers from histopathology whole slide images. However, explainable artificial intelligence approaches widely used in digital pathology, such as attention heatmaps and class activation mapping, offer only limited interpretability regarding the features captured by classifiers. Here, we present MoPaDi (Morphing histoPathology Diffusion), a framework for generating counterfactual explanations for histopathology images that reveal which morphological or style features drive classifier predictions. MoPaDi combines diffusion autoencoders with task-specific multiple instance learning classifiers to manipulate images and flip predictions by modifying relevant features. We evaluated the framework on multiple datasets spanning colorectal, breast, liver, and lung cancers, including tissue type, cancer subtype, and biomarker (microsatellite instability) classification tasks. We assessed counterfactual explanations through quantitative analyses, pathologists evaluations, and independent foundation model-based classifiers. We found that MoPaDi was able to generate realistic counterfactual histopathology images, enabling pathologists to identify morphological features associated with the change in model predictions. Unlike conventional reviews of highly attended regions typical in digital pathology, MoPaDi explanations enabled pathologists to directly identify morphological features driving the classifiers predictions from a limited number of top-contributing tiles. Consistent with the literature, our biomarker classifier associated high microsatellite instability with mucinous differentiation, glandular patterns, and lymphocytic infiltration. Furthermore, MoPaDi revealed that changes in classifier predictions were mainly driven by morphological alterations rather than staining differences. Overall, MoPaDi is a practical framework for counterfactual explanations in computational pathology that reveals model-specific drivers of classification and increases trust in deep learning models.

bioinformatics↗

Multimodal histopathologic models stratify hormone receptor-positive early breast cancer

For patients with hormone receptor-positive, early breast cancer without HER2 amplification, multigene expression assays including Oncotype DX (R) recurrence score (RS) have been clinically validated to identify patients who stand to derive added benefit from adjuvant cytotoxic chemotherapy. However, cost and turnaround time have limited its global adoption despite recommendation by practice guidelines. We investigated if routinely available hematoxylin and eosin (H&E)-stained pathology slides could act as a surrogate triaging data substrate by predicting RS using machine learning methods. We trained and validated a multimodal transformer model, Orpheus, using 6,203 patients across three independent cohorts, taking both H&E images and their corresponding synoptic text reports as input. We showed accurate inference of recurrence score from whole-slide images (r = 0.63 (95% C.I. 0.58 - 0.68); n = 1,029), the raw text of their corresponding reports (r = 0.58 (95% C.I. 0.51 - 0.64); n = 972), and their combination (r = 0.68 (95% C.I. 0.64 - 0.73); n = 964) as measured by Pearsons correlation. To predict high-risk disease (RS>25), our model achieved an area under the receiver operating characteristic curve (AUROC) of 0.89 (95% C.I. 0.83 - 0.94), and area under the precision recall curve (AUPRC) of 0.64 (95% C.I. 0.60 - 0.82), compared to 0.49 (95% C.I. 0.36 - 0.64) for an existing nomogram based on clinical and pathologic features. Moreover, our model generalizes well to external international cohorts, effectively identifying recurrence risk (r = 0.61, p < 10-4, n = 452; r = 0.60, p < 10-4, n = 575) and high-risk status (AUROC = 0.80, p < 10-4, AUPRC = 0.68, p < 10-4, n = 452; AUROC = 0.83, p < 10-4, AUPRC = 0.73, p < 10-4, n = 575) from whole-slide images. Probing the biologic underpinnings of the model decisions uncovered tumor cell size heterogeneity, immune cell infiltration, a proliferative transcription program, and stromal fraction as correlates of higher-risk predictions. We conclude that at an operating point of 94.4% precision and 33.3% recall, this model could help increase global adoption and shorten lag between resection and adjuvant therapy.

pathology↗

Validating a low-cost, open-source, locally manufactured workstation and computational pipeline for automated histopathology evaluation using deep learning

Deployment and access to state-of-the-art diagnostic technologies remains a fundamental challenge in providing equitable global cancer care to low-resource settings. The expansion of digital pathology in recent years and its interface with computational biomarkers provides an opportunity to democratize access to personalized medicine. Here we describe a low-cost platform for digital side capture and computational analysis composed of open-source components. The platform provides low-cost ($200) digital image capture from glass slides and is capable of real-time computational image analysis using an open-source deep learning (DL) algorithm and Raspberry Pi ($35) computer. We validate the performance of deep learning models performance using images captured from the open-source workstation and show similar model performance when compared against significantly more expensive standard institutional hardware.

pathology↗

Latent transcriptional programs reveal histology-encoded tumor features spanning tissue origins

Precision medicine in cancer treatment depends on deciphering tumor phenotypes to reveal the underlying biological processes. Molecular profiles, including transcriptomics, provide an information-rich tumor view, but their high-dimensional features and assay costs can be prohibitive for clinical translation at scale. Recent studies have suggested jointly leveraging histology and genomics as a strategy for developing practical clinical biomarkers. Here, we use machine learning techniques to identify de novo latent transcriptional processes in squamous cell carcinomas (SCCs) and to accurately predict their activity levels directly from tumor histology images. In contrast to analyses focusing on pre-specified, individual genes or sample groups, our latent space analysis reveals sets of genes associated with both histologically detectable features and clinically relevant processes, including immune response, collagen remodeling, and fibrosis. The results demonstrate an approach for discovering clinically interpretable histological features that indicate complex, potentially treatment-informing biological processes.

cancer biology↗

Self-supervised deep learning for pan-cancer mutation prediction from histopathology

The histopathological phenotype of tumors reflects the underlying genetic makeup. Deep learning can predict genetic alterations from tissue morphology, but it is unclear how well these predictions generalize to external datasets. Here, we present a deep learning pipeline based on self-supervised feature extraction which achieves a robust predictability of genetic alterations in two large multicentric datasets of seven tumor types.

cancer biology↗

Adversarial attacks and adversarial robustness in computational pathology

Artificial Intelligence (AI) can support diagnostic workflows in oncology by aiding diagnosis and providing biomarkers. AI applications are therefore expected to evolve from academic prototypes to commercial products in the coming years. However, AI applications are vulnerable to adversarial attacks, such as malicious interference with test data aiming to cause misclassifications. Therefore, it is essential for the use of AI-based diagnostic devices to secure them against such attacks before widespread use. Unfortunately, no resistant systems exist in computational pathology so far. To address this problem, we investigate the susceptibility of convolutional neural networks (CNNs) to multiple types of white- and black-box attacks. We demonstrate that both attacks can easily confuse CNNs in clinically relevant pathology tasks and impair classification performance. Classical adversarially robust training and dual batch normalization (DBN) are possible mitigation strategies but require precise knowledge of the type of attack used in the inference. We demonstrate that vision transformers (ViTs) perform equally well compared to CNNs at baseline and are orders of magnitude more robust to different types of white-box and black-box attacks. At a mechanistic level, we show that this is associated with a more robust latent representation of clinically relevant categories in ViTs compared to CNNs. Our results are in line with previous theoretical studies. We show that ViTs are robust learners in computational pathology. This implies that large-scale rollout of AI models in computational pathology should rely on ViTs rather than CNN-based classifiers to provide inherent protection against adversaries.

bioinformatics↗

Can the Kuznetsov Model Replicate and Predict Cancer Growth in Humans?

Several mathematical models to predict tumor growth over time have been developed in the last decades. A central aspect of such models is the interaction of tumor cells with immune effector cells. The Kuznetsov model (Kuznetsov et al. (1994), Bulletin of Mathematical Biology, vol. 56, no. 2, pp. 295-321) is the most prominent of these models and has been used as a basis for many other related models and theoretical studies. However, none of these models have been validated with large-scale real-world data of human patients treated with cancer immunotherapy. In addition, parameter estimation of these models remains a major bottleneck on the way to model-based and data-driven medical treatment. In this study, we quantitatively fit Kuznetsovs model to a large dataset of 1472 patients, of which 210 patients have more than six data points, by estimating the model parameters of each patient individually. We also conduct a global practical identifiability analysis for the estimated parameters. We thus demonstrate that several combinations of parameter values could lead to accurate data fitting. This opens the potential for global parameter estimation of the model, in which the values of all parameters are fixed for all patients. Furthermore, by omitting the last two or three data points, we show that the model can be extrapolated and predict future tumor dynamics. This paves the way for a more clinically relevant application of mathematical tumor modeling, in which the treatment strategy could be adjusted in advance according to the models future predictions.

bioinformatics↗

The future of artificial intelligence in digital pathology - results of a survey across stakeholder groups

Artificial intelligence (AI) provides a powerful tool to extract information from digitized histopathology whole slide images. In the last five years, academic and commercial actors have developed new technical solutions for a diverse set of tasks, including tissue segmentation, cell detection, mutation prediction, prognostication and prediction of treatment response. In the light of limited overall resources, it is presently unclear for researchers, practitioners and policymakers which of these topics are stable enough for clinical use in the near future and which topics are still experimental, but worth investing time and effort into. To identify potentially promising applications of AI in pathology, we performed an anonymous online survey of 75 computational pathology domain experts from academia and industry. Participants enrolled in 2021 were queried about their subjective opinion on promising and appealing sub-fields of computational pathology with a focus on solid tumors. The results of this survey indicate that the prediction of treatment response directly from routine pathology slides is regarded as the most promising future application. This item was ranked highest in the overall analysis and in sub-groups by age and professional background. Furthermore, prediction of genetic alterations, gene expression and survival directly from routine pathology images scored consistently high across subgroups. Together, these data demonstrate a possible direction for the development of computational pathology systems in clinical, academic and industrial research in the near future.

pathology↗

DeepMed: A unified, modular pipeline for end-to-end deep learning in computational pathology

The interpretation of digitized histopathology images has been transformed thanks to artificial intelligence (AI). End-to-end AI algorithms can infer high-level features directly from raw image data, extending the capabilities of human experts. In particular, AI can predict tumor subtypes, genetic mutations and gene expression directly from hematoxylin and eosin (H&E) stained pathology slides. However, existing end-to-end AI workflows are poorly standardized and not easily adaptable to new tasks. Here, we introduce DeepMed, a Python library for predicting any high-level attribute directly from histopathological whole slide images alone, or from images coupled with additional meta-data (https://github.com/KatherLab/deepmed). Unlike earlier computational pipelines, DeepMed is highly developer-friendly: its structure is modular and separates preprocessing, training, deployment, statistics, and visualization in such a way that any one of these processes can be altered without affecting the others. Also, DeepMed scales easily from local use on laptop computers to multi-GPU clusters in cloud computing services and therefore can be used for teaching, prototyping and for large-scale applications. Finally, DeepMed is user-friendly and allows researchers to easily test multiple hypotheses in a single dataset (via cross-validation) or in multiple datasets (via external validation). Here, we demonstrate and document DeepMeds abilities to predict molecular alterations, histopathological subtypes and molecular features from routine histopathology images, using a large benchmark dataset which we release publicly. In summary, DeepMed is a fully integrated and broadly applicable end-to-end AI pipeline for the biomedical research community.

cancer biology↗

Swarm learning for decentralized artificial intelligence in cancer histopathology

Artificial Intelligence (AI) can extract clinically actionable information from medical image data. In cancer histopathology, AI can be used to predict the presence of molecular alterations directly from routine histopathology slides. However, training robust AI systems requires large datasets whose collection faces practical, ethical and legal obstacles. These obstacles could be overcome with swarm learning (SL) where partners jointly train AI models, while avoiding data transfer and monopolistic data governance. Here, for the first time, we demonstrate the successful use of SL in large, multicentric datasets of gigapixel histopathology images comprising over 5000 patients. We show that AI models trained using Swarm Learning can predict BRAF mutational status and microsatellite instability (MSI) directly from hematoxylin and eosin (H&E)-stained pathology slides of colorectal cancer (CRC). We trained AI models on three patient cohorts from Northern Ireland, Germany and the United States of America and validated the prediction performance in two independent datasets from the United Kingdom using SL-based AI models. Our data show that SL enables us to train AI models which outperform most locally trained models and perform on par with models which are centrally trained on the merged datasets. In addition, we show that SL-based AI models are data efficient and maintain a robust performance even if only subsets of local datasets are used for training. In the future, SL can be used to train distributed AI models for any histopathology image analysis tasks, overcoming the need for data transfer and without requiring institutions to give up control of the final AI model.

bioinformatics↗

Improving mathematical models of cancer by including resistance to therapy: a study in non-small cell lung cancer

We examined a dataset of 590 Non-Small Cell Lung Cancer patients treated with either chemotherapy or immunotherapy using a game-theoretic model that includes both the evolution of therapy resistance and a cost of resistance. We tested whether the game-theoretic model provides a better fit than classical mathematical models of population growth (exponential, logistic, classic Bertalanffy, general Bertalanffy, Gompertz, general Gompertz). To our knowledge, this is the first time a large clinical patient cohort (as opposed to only in-vitro data) has been used to apply a game-theoretic cancer model. The game-theoretic model provided a better fit to the tumor dynamics of the 590 Non-Small Cell Lung Cancer patients than any of the non-evolutionary population growth models. This was not simply due to having more parameters in the game-theoretic model. The game-theoretic model was seemingly able to fit more accurately patients whose tumor burden exhibit a U-shaped trajectory over time. We explained how this game-theoretic model provides predictions of future tumor growth based on just a few initial measurements. Using the estimates for treatment-specific parameters, we then explored alternative treatment protocols and their expected impact on tumor growth and patient outcome. As such, the model could possibly be used to suggest patient-specific optimal treatment regimens with the goal of minimizing final tumor burden. Therapeutic protocols based on game-theoretic modeling can help to predict tumor growth, and could potentially improve patient outcome in the future. The model invites evolutionary therapies that anticipate and steer the evolution of therapy resistance.

cancer biology↗

Classical Mathematical Models for Prediction of Response to Chemotherapy and Immunotherapy

Classical mathematical models of tumor growth have shaped our understanding of cancer and have broad practical implications for treatment scheduling and dosage. However, even the simplest textbook models have been barely validated in real world-data of human patients. In this study, we fitted a range of differential equation models to tumor volume measurements of patients undergoing chemotherapy or cancer immunotherapy for solid tumors. We used a large dataset of 1472 patients with three or more measurements per target lesion, of which 652 patients had six or more data points. We show that the early treatment response shows only moderate correlation with the final treatment response, demonstrating the need for nuanced models. We then perform a head-to-head comparison of six classical models which are widely used in the field: the Exponential, Logistic, Classic Bertalanffy, General Bertalanffy, Classic Gompertz and General Gompertz model. Several models provide a good fit to tumor volume measurements, with the Gompertz model providing the best balance between goodness of fit and number of parameters. Similarly, when fitting to early treatment data, the general Bertalanffy and Gompertz models yield the lowest mean absolute error to forecasted data, indicating that these models could potentially be effective at predicting treatment outcome. In summary, we provide a quantitative benchmark for classical textbook models and state-of-the art models of human tumor growth. We publicly release an anonymized version of our original data, providing the first benchmark set of human tumor growth data for evaluation of mathematical models. Author SummaryMathematical oncology uses quantitative models for prediction of tumor growth and treatment response. The theoretical foundation of mathematical oncology is provided by six classical mathematical models: the Exponential, Logistic, Classic Bertalanffy, General Bertalanffy, Classic Gompertz and General Gompertz model. These models have been introduced decades ago, have been used in thousands of scientific articles and are part of textbooks and curricula in mathematical oncology. However, these models have not been systematically tested in clinical data from actual patients. In this study, we have collected quantitative tumor volume measurements from thousands of patients in five large clinical trials of cancer immunotherapy. We use this dataset to systematically investigate how accurately mathematical models can describe tumor growth, showing that there are pronounced differences between models. In addition, we show that two of these models can predict tumor response to immunotherapy and chemotherapy at later time points when trained on early tumor growth dynamics. Thus, our article closes a conceptual gap in the literature and at the same time provides a simple tool to predict response to chemotherapy and immunotherapy on the level of individual patients.

cancer biology↗

Benchmarking artificial intelligence methods for end-to-end computational pathology

Artificial intelligence (AI) can extract subtle visual information from digitized histopathology slides and yield scientific insight on genotype-phenotype interactions as well as clinically actionable recommendations. Classical weakly supervised pipelines use an end-to-end approach with residual neural networks (ResNets), modern convolutional neural networks such as EfficientNet, or non-convolutional architectures such as vision transformers (ViT). In addition, multiple-instance learning (MIL) and clustering-constrained attention MIL (CLAM) are being used for pathology image analysis. However, it is unclear how these different approaches perform relative to each other. Here, we implement and systematically compare all five methods in six clinically relevant end-to-end prediction tasks using data from N=4848 patients with rigorous external validation. We show that histological tumor subtyping of renal cell carcinoma is an easy task which approaches successfully solved with an area under the receiver operating curve (AUROC) of above 0.9 without any significant differences between approaches. In contrast, we report significant performance differences for mutation prediction in colorectal, gastric and bladder cancer. Weakly supervised ResNet-and ViT-based workflows significantly outperformed other methods, in particular MIL and CLAM for mutation prediction. As a reason for this higher performance we identify the ability of ResNet and ViT to assign high prediction scores to highly informative image regions with plausible histopathological image features. We make all source codes publicly available at https://github.com/KatherLab/HIA, allowing easy application of all methods on any end-to-end problem in computational pathology.

bioinformatics↗