A comprehensive investigation of statistical and machine learning approaches for predicting complex human diseases on genomic variants

Abstract

BackgroundQuantifying an individuals risk for common diseases is an important goal of precision health. The polygenic risk score (PRS), which aggregates multiple risk alleles of candidate diseases, has emerged as a standard approach for identifying high-risk individuals. A variety of tools have been developed to implement PRS. However, benchmarks for comparatively evaluating the performance of these different methods and for assessing their potential to guide future clinical applications are lacking. ResultsWe systematically validated and compared thirteen statistical methods, five machine learning models and two ensemble models using simulated data, twenty-two common diseases with internal training sets and four diseases with external summary statistics from the UK Biobank resource. The effects of disease heritability, single nucleotide polymorphism (SNP) effect size and sample size are evaluated using simulated data. We also investigated the correlations between methods and their standard deviations of different diseases. ConclusionsIn general, statistical methods outperform machine learning models, and ensemble models, such as Super Learner, generally perform the best for most situations. We observed the correlations were relatively high if the methods were from the same category and the external summary statistics from large cohort GWAS could decrease the standard deviation of method correlations. By varying three factors in the simulated data, we also identified that disease heritability had a strong effect on the predictive performance of individual methods. Both the number and effect sizes of risk SNPs are important; and while sample size strongly influences the performance of machine learning models, but not statistical methods.