Model-based detection and analysis of introgressed Neanderthal ancestry in modern humans

Abstract

Genetic evidence has revealed that the ancestors of modern human populations outside of Africa and their hominin sister groups, notably the Neanderthals, exchanged genetic material in the past. The distribution of these introgressed sequence-tracts along modern-day human genomes provides insight into the ancient structure and migration patterns of these archaic populations. Furthermore, it facilitates studying the selective processes that lead to the accumulation or depletion of introgressed genetic variation. Recent studies have developed methods to localize these introgressed regions, reporting long regions that are depleted of Neanderthal introgression and enriched in genes, suggesting negative selection against the Neanderthal variants. On the other hand, enriched Neanderthal ancestry in hair- and skin-related genes suggests that some introgressed variants facilitated adaptation to new environments. Here, we present a model-based method called diCal-admix and apply it to detect tracts of Neanderthal introgression in modern humans. We demonstrate its efficiency and accuracy through extensive simulations. We use our method to detect introgressed regions in modern human individuals from the 1000 Genomes Project, using a high coverage genome from a Neanderthal individual from the Altai mountains as reference. Our introgression detection results and findings concerning their functional implications are largely concordant with previous studies, and are consistent with weak selection against Neanderthal ancestry. We find some evidence that selection against Neanderthal ancestry was due to higher genetic load in Neanderthals, resulting from small effective population size, rather than Dobzhansky-Muller incompatibilities. Finally, we investigate the role of the X-chromosome in the divergence between Neanderthals and modern humans.