R2ROC: An efficient method of comparing two or more correlated AUC from out-of-sample prediction using polygenic scores

Abstract

Polygenic risk scores (PRSs) enable early prediction of disease risk. Evaluating PRS performance for binary traits commonly relies on the area under the receiver operating characteristic curve (AUC). However, the widely used DeLongs method for comparative significance tests suffer from limitations, including computational time and the lack of a one-to-one mapping between test statistics based on AUC and R2. To overcome these limitations, we propose a novel approach that leverages the Delta method to derive the variance and covariance of AUC values, enabling a comprehensive and efficient comparative significance test. Our approach offers notable advantages over DeLongs method, including reduced computation time (up to 150-fold), making it suitable for large-scale analyses and ideal for integration into machine learning frameworks. Furthermore, our method allows for a direct one-to-one mapping between AUC and R2 values for comparative significance tests, providing enhanced insights into the relationship between these measures and facilitating their interpretation. We validated our proposed approach through simulations and applied it to real data comparing PRSs for diabetes and coronary artery disease (CAD) prediction in a cohort of 28,880 European individuals. The PRSs were derived using genome-wide association study summary statistics from two distinct sources. Our approach enabled a comprehensive and informative comparison of the PRSs, shedding light on their respective predictive abilities for diabetes and CAD. This advancement contributes to the assessment of genetic risk factors and personalized disease prediction, supporting better healthcare decision-making.