Robust taxonomic classification in gut and vaginal microbiomes demonstrated through benchmarking with age-specific synthetic communities

Abstract

Accurate taxonomic profiling of human microbiomes is essential for advancing research and understanding the complex role microbial communities play in human health. When using shotgun metagenomics, the sequencing data is analyzed through metagenomic pipelines, which incorporate various open-source tools and classify microbes based on matched paired-end DNA reads. However, differences in sequencing and computational approaches can produce substantially different microbiome profiles from the same sample, making validation critical. One approach for validation is benchmarking with realistic mock communities, but this remains relatively rare. Additionally, existing benchmarks often overlook microbiome variability across life stages and body sites, limiting their clinical and research utility. Here, we developed age- and body site-stratified synthetic metagenomes, enabling context-aware benchmarking of microbiome pipelines. Using novelty-based sampling to prioritize microbial diversity and minimize redundancy among selected samples, we selected 300 representative, real biological samples spanning six categories: adult, child, toddler, and infant (>6 months and <6 months) gut samples, as well as adult vaginal samples. We validated three pipelines, Tiny Health's proprietary Metagenomic Classifier v2 (THMCv2), MetaPhlAn4, and Kraken2+Bracken, using precision, recall, F1 score, and area under the precision-recall curve (AUPR) across age groups and sample types. THMCv2 demonstrated higher recall and F1 scores, detecting more taxa across sample types and ages, while MetaPhlAn4 achieved the highest precision. THMCv2 also achieved the highest area under the precision-recall curve, reflecting peak performance across both abundant and rare species. When analyses were weighted by abundance, THMCv2 and MetaPhlAn4 each characterized the mock community nearly perfectly. Errors for THMCv2 were largely restricted to very low-abundance taxa (<0.001%), whereas MetaPhlAn4 occasionally produced false positives for higher-abundance taxa. Species-level analyses of clinically relevant microbes confirmed these patterns, with THMCv2 demonstrating higher sensitivity, MetaPhlAn4 higher specificity, and Kraken2 lower overall performance. These results demonstrate clear precision-recall trade-offs in metagenomic profiling. This benchmarking framework provides a reproducible approach for evaluating pipeline performance across diverse microbiome contexts and life stages.