When species trees disagree: an approach consistent with the coalescent that quantifies phylogenomic support for contentious relationships

Abstract

Phylogenies inferred using both concatenation- and coalescent-based analyses typically render highly congruent trees. However, when they disagree, they often differ with respect to historically contentious and evolutionarily important relationships. These relationships may also involve etiolated lineages where increased sampling is not possible. Recently, methods aimed at interrogating single relationships or trees have emerged as promising investigative tools to examine these cases. Although recent methods such as "Edge-based Phylogenomic Support analYsis" (EPSY) led to insights into both systematic error and real biological signal, whether they are consistent with the coalescent in cases with high Incomplete Lineage Sorting (ILS) has yet to be characterized. Here, we use simulations and an empirical dataset to test the performance of EPSY, concatenation, and coalescent-based summary analyses under high levels of ILS. We focused on high-ILS scenarios because these represent the typical difficult cases that researchers often face due to the prevalence of ILS in phylogenomic datasets. ILS is known to be a major cause of phylogenomic conflict, which confounds many biological conclusions that depend on a resolved phylogeny, such as inferring ancestral character states, biogeographic reconstructions, and domestication histories. Our study found that EPSY was consistent with the coalescent in a high-ILS empirical dataset. In high-ILS simulations EPSY infers the correct edge more than half the time, whereas coalescent based methods and concatenation methods inferred the actual tree 37.8% and 25% of the time, respectively. All methods have conditions under which they generate the most accurate inferences. Given the levels of ILS in simulations, 26.2% of the time no method recovered the true tree. This zone where no current method can infer the true topology is likely due to properties of the species tree, such as the length of internal edges adjacent to a conflict and/or the length of the shortest branch. Nevertheless, the EPSY approach proves to be a valuable complement to phylogenomic analyses for interrogating regions of the tree with conflicting hypotheses generated from past studies or alternative inference methods. Our analyses highlight that robust phylogenetic trees may not be possible under some scenarios regardless of method and data source.