Extracting diamonds: Identifiability of 4-node cycles in level-1 phylogenetic networks under a pseudolikelihood coalescent model

Abstract

Phylogenetic networks encode a broader picture of evolution by the inclusion of reticulate processes such as hybridization, introgression or horizontal gene transfer. Each reticulation event is represented by a "hybridization cycle". Here, we investigate the statistical identifiability of the position of the hybrid node in a 4-node hybridization cycle in a semi-directed level-1 phylogenetic network. That is, we investigate if our model is able to detect the correct placement of the hybrid node in the hybridization cycle from concordance factors as data. While generic identifiability is easily attained under non-restrictive assumptions such as t [isin] (0, {infty}) for all branches and{gamma} [isin] (0, 1) for the inheritance probability of the hybrid edges, simulations show that accurate detection of these cycles can be complicated by inadequate sampling, small sample size or gene tree estimation error. We identify practical advice for evolutionary biologists on best sampling strategies to improve the detection of this type of hybridization cycle.