Matched full-UDG and non-UDG ancient DNA libraries reveal trade-offs in post-mortem damage correction for imputation and kinship inference

Abstract

Ancient DNA studies increasingly combine full uracil-DNA glycosylase-treated, full-UDG, and non-UDG libraries, but which computational damage correction to apply before imputation and kinship analysis remains unsettled. We compared terminal trimming, base-quality rescaling and known-SNP masking in matched full-UDG and non-UDG libraries from the same extracts of two medieval Mongolian individuals. All methods reduced or reversed the difference in mean alternative-allele fraction between damage-prone and transversion SNPs but retained different proportions of covered sites. In non-UDG libraries, masking and rescaling within five bases of each fragment end produced similar cross-library non-reference discordance, NRD, while retaining 92% and 99.3% of covered sites, respectively. In these 3'-biased libraries, ten-base 3'-only trimming retained more covered sites at lower observed discordance than five-base-per-end symmetric trimming. ancIBD inferred widespread sharing of one identity-by-descent chromosome copy, IBD1, across correction methods. Uncorrected non-UDG data shifted TKGWV2 toward second-degree estimates, whereas corrected mixed-library comparisons supported first-degree relatedness. READv2 and the low rate of opposite-homozygote sites, IBS0, supported a parent-offspring relationship. Mitochondrial data and genetic sex favoured ORT16 as mother and ORT15 as son. No method performed best across all measures, so the choice depends on whether residual damage, site retention or the downstream analysis matters most.