Inferring inter- and intrachromosomal Dobzhansky-Muller incompatibilities from imbalanced segregation of recombinant haplotypes in hybrid populations

Abstract

Dobzhansky-Muller incompatibilities (DMIs) are a major component of reproductive isolation between species. DMIs imply negative epistasis, and are exposed when two diverged populations hybridize. Mapping the locations of DMIs has largely relied on classical genetic mapping. Approaches to date are hampered by low power and the challenge of identifying DMI loci on the same chromosome, because strong initial linkage of parental haplotypes weakens statistical tests. Here, we propose new statistics to infer negative epistasis from haplotype frequencies in hybrid populations. When two divergent populations hybridize, the variance in heterozygosity at two loci decreases faster with time at DMI loci than at random pairs of loci. When two populations hybridize at near-even admixture proportions, the deviation of the observed variance from its expectation becomes negative for the DMI pair. This negative deviation enables us to detect intermediate to strong negative epistasis both within and between chromosomes. In practice, the detection window in hybrid populations depends on the demographic scenario, the recombination rate, and the strength of epistasis. When the initial proportion of the two parental populations is uneven, only strong DMIs can be detected with our method unless migration prevents parental haplotypes from being lost. We use the new statistics to infer candidate DMIs from three hybrid populations of swordtail fish. We identify numerous new DMI candidates, some of which are inferred to interact with several loci within and between chromosomes. Moreover, we discuss our results in the context of an expected enrichment in intrachromosomal over interchromosomal DMIs. Author SummaryGenetic incompatibility in the form of (Bateson-)Dobzhansky-Muller incompatibilities (DMIs) is an important component of reproductive isolation between species. However, the evolutionary role of DMIs during the process of speciation is contentious. DMIs occur when two or more genetic variants interact to reduce their carriers fitness. Once recombination combines incompatible variants in hybrids, selection acts to remove these variants from the population. One step towards addressing the evolutionary role of DMIs is to quantify the prevalence of DMIs in incipient and hybridizing species. Here, we present statistics that are sensitive to the resulting recombinant imbalance and that can indicate the location of DMIs in hybrid genomes in various demographic scenarios. We use simulations to show that the time window during which a DMI is detectable depends on its genomic location, the severity of the DMI, and the populations demography. Importantly, our statistic distinguishes genetic associations arising due to physical linkage from those arising due to gene interactions, which allows for the inference of both inter- and intrachromosomal DMIs. Applying our statistics to three hybrid populations of swordtail fish, we confirm previously known DMIs and identify new candidate incompatibilities.