A method for identifying spatially divergent selection in structured populations.

Abstract

Species occupy diverse, heterogeneous environments, which expose populations to spatially varied selective pressures. Populations in different environments can diverge due to local adaptation. However, neutral evolution can also drive population divergence. Thus, testing for local adaptation requires a neutral baseline for population differentiation. The classical QST -FST comparison was developed for this purpose. Yet, QST -FST frequently fails to account for the complexities of population structure because the theory underlying this comparison assumes that all subpopulations are equally related, resulting in inflated false positive rates in metapopulations that deviate from the island model. To address this limitation we use estimates of between- and within-population relatedness to model population structure. Using those relatedness matrices, we infer the between- and within-population ancestral additive genetic variances under a mixed-effects model. Under neutrality, these inferred variances are expected to be equal. We propose here a test to detect selection based on the comparison of these two estimates of the ancestral variance and we compare its performance with earlier solutions. We find our method is well calibrated across various population structures and has high power to detect adaptive divergence. Author summaryPopulations of the same species often face different environmental pressures, driving them to adapt locally. However, even in the absence of adaptation, subpopulations can diverge due to random genetic drift and limited migration. Distinguishing between adaptive evolution and random divergence is a central challenge in evolutionary biology. Traditional methods, such as QST -FST comparison, assume equal relatedness among subpopulations--a simplification that rarely holds in complex real-world scenarios, leading to flawed conclusions. To overcome this limitation, we developed a novel method that incorporates genetic relatedness among subpopulations, leveraging quantitative genetic theory to estimate ancestral additive genetic variances. Our approach provides a powerful tool for testing local adaptation, reliably distinguishing adaptive divergence from drift across a variety of population structures.