bioRxiv2017
Machine learning (ML) encompasses numerous algorithms that aim at discovering complex patterns between elements within large data using limited prior assumptions or modeling. However, some scientific disciplines still produce small data sets: in particular, empirical studies that try to find the mutations responsible for complex phenotypes are often limited to very small sample sizes (n), while scanning a large number of amino acid sites (p) in a proteome. To date, little is known on how ML performs in this type of so-called \"large p, small n\" problem. To address this question, we evaluated the performance of two general ML classifiers, adaptive boosting (AB) and random forest, on two data sets. To assess the impact of proteome size, we contrasted a small (viral) genome with a larger (bacterial) one. To analyze large proteomes, we further developed a chunking algorithm, and introduce a repeated random forest (RRF) algorithm that stabilizes model predictions. With the influenza data, we were able to rediscover amino acid sites experimentally implicated in three different complex phenotypes (infectivity, transmissibility, and pathogenicity). Results for the larger proteome, pertaining to three types of drug resistance (Ciprofloxacin, Ceftazidime, and Gentamicin), were more nuanced, with RRF making more sensible pre-dictions, with smaller errors rates, than AB. Furthermore, we show that chunking improved runtimes by an order of magnitude and may increase sensitivity of the predictions. Altogether, we demonstrate that ML algorithms can be used to identify genetic determinants in small proteomes (viruses), even with small numbers of individuals. We further show that even if the size of bacterial proteomes pushes AB to its limits in the context of small n, RRF may deserve more scrutiny, which should be facilitated by the plummeting costs of sequencing and, more critically, by phenotyping large cohorts of individuals.\n\nAuthor SummaryFinding the genetic determinants of a phenotype is typically performed by testing for an association between a particular allele and a trait, carrying out the testing over a large number of loci in a large cohort of individuals, itself divided into two subsets of individuals: those who have the trait (cases), and those who do not (controls). However, recruiting large cohorts can be problematic in some experimental fields, while using genotypic information rather than complete genomes can miss some mutations. To address these issues, we implemented two machine learning (ML) algorithms, tweaked for analyzing large genomes and providing stable results. The analysis of a small viral genome, for which genetic determinants of three phenotypes are already known, showed that our approach can rediscover known mutations, almost irrespective of the ML algorithm used. However, the analysis of a larger bacterial genome, for which genetic determinants of three phenotypes are unknown, suggested that the simpler of our modified algorithms performed better, returning more sensitive predictions with lower error rates. This work demonstrates the feasibility of finding genetic determinants of complex phenotypes based on a small number of complete genomes.