DeepPROTECTNeo: A Deep learning-based Personalized and RV-guided Optimization tool for TCR Epitope interaction using Context-aware Transformers

Abstract

Background: The development of personalized cancer vaccines relies on accurately identifying neoepitopes capable of eliciting strong immune responses. T cell receptor (TCR)-epitope interactions are fundamental to cancer immunotherapy. Traditional computational approaches focus primarily on epitope-major histocompatibility complex (MHC) binding, often overlooking the critical contribution of TCR binding. Furthermore, the clinical applicability of existing methods is constrained by fragmented pipelines that require separate workflows for variant calling, HLA typing, and independent peptide-MHC (pMHC) or peptide-TCR (pTCR) evaluation stages. Results: We present DeepPROTECTNeo, a unified deep learning framework that integrates genomic variant detection, HLA typing, high-affinity pMHC binding prediction, variant-driven TCR repertoire mining, followed by a hybrid transformer-Convolutional Neural Network dual-branch feature extractor with an explicit cross-attention-based deep learning model for TCR-epitope binding prediction. Our reverse vaccinology-inspired biologically informed architecture integrates Bidirectional Long short-term memory (Bi-LSTM) sequence features, convolutional-attention physicochemical/evolutionary descriptors via gated fusion, and TCR numbered contextual embeddings to enable residue-level interpretable modelling. Under a strict TCR-split strategy, it achieved a mean AUROC of 0.7856 and AUPRC of 0.7932 outperforming six state-of-the-art predictors by 4-5% with tight inter-fold stability. The architecture maintains high robustness against structural hard negatives and imbalanced datasets, successfully recovering 18 of 34 validated high-affinity neoepitopes from a patient-specific cancer cohort. Conclusions: Experiments results demonstrate that DeepPROTECTNeo is a powerful, reliable end-to-end neoantigen prioritization framework that effectively models complex TCR-epitope interfaces directly from clinical sequencing data, providing a robust interpretable foundation to accelerate personalized cancer immunotherapy.