Predicting Pre-treatment Resistance or Post-treatment Effect? A Systematic Benchmarking of Single-Cell Drug Response Models
Intratumoral heterogeneity drives variable drug responses in cancer. Single-cell RNA sequencing (scRNA-seq) enables characterization of such heterogeneity and prediction of drug response at single-cell resolution. Accordingly, various computational models have been developed to infer drug response from scRNA-seq data. However, their performance, robustness, and generalizability across different biological contexts remain insufficiently evaluated. To address this gap, we benchmarked representative single-cell drug response prediction models using 26 curated datasets comprising over 760,000 cells across 12 cancer types and 21 therapeutic agents. We constructed balanced and imbalanced scenarios to reflect realistic drug-response label distributions. To address the lack of ground-truth labels in conventional scRNA-seq datasets, we incorporated lineage-tracing data with experimentally validated drug-response annotations, enabling evaluation in a clinically relevant pre-treatment prediction setting. Our results show that prediction performance was markedly higher in cell lines than in tissue samples. Under imbalanced conditions, most methods exhibited sharp performance declines, whereas scDEAL demonstrated the highest robustness. Independent validation using an in-house pancreatic ductal adenocarcinoma dataset further confirmed scDEALs robustness and ability to capture biologically meaningful state transitions. Label-substitution experiment revealed that this robustness was partially driven by the models specific training-label construction. However, benchmarking with lineage-tracing data revealed a fundamental limitation: most models capture drug-induced transcriptional changes but struggled to predict intrinsic resistance before treatment. In summary, our study defines the performance boundaries of current approaches and highlights their limitations in addressing intratumoral heterogeneity, class imbalance, and intrinsic resistance prediction, emphasizing the need for the next-generation single-cell drug response models with stronger clinical relevance.