A Rarefaction Approach to Identify Local Introgression in a Three Population Tree

Abstract

The $D$ statistic, also known as the $ABBA-BABA$ statistic, is widely used to detect the presence of archaic genome-wide introgression between two non-sister taxa. $D$ counts the imbalance between the number of biallelic sites where either the second and third taxa (ABBA site) share the derived allele or the first and third taxa (BABA site) share the derived allele in a four taxa tree. Here, the fourth taxon acts as an outgroup to determine the ancestral allele. When there is no introgression, these counts are expected to be equal, and a discordance between counts suggests introgression from the third taxon into either the first or second. D is limited to the detection of genome-wide introgression and exhibits a high false-positive rate when applied to smaller genomic segments. Here, we present a new method, D STatistic with Allelic Rarefaction ($\dstar$), to address these limitations. $\dstar$ uses multiple lineages and does not require an outgroup to calculate the imbalance between the number of alleles found exclusively in the second and third taxa and the number of alleles found exclusively in the first and third taxa. $\dstar$ employs a rarefaction technique to correct for unequal sample-size and allows multiallelic sites. We use simulations to show that $\dstar$ has better precision and recall for detecting introgressed segments of DNA when compared to other methods. We conclude by recovering Denisovan DNA related to immune function in modern day Papuans. Precompiled executables, the manual, source code, and simulation and analysis scripts used in this study can be found at \url{https://github.com/TQ-Smith/DSTAR}