Search bioRxiv⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/372680

Assessing taxonomic metagenome profilers with OPAL

Meyer, F.·Bremges, A.·Belmann, P.·Janssen, S.·McHardy, A. C.·Koslicki, D.
View original source ↗Export metadata ↓

Abstract

Taxonomic metagenome profilers predict the presence and relative abundance of microorganisms from shotgun sequence samples of DNA isolated directly from a microbial community. Over the past years, there has been an explosive growth of software and algorithms for this task, resulting in a need for more systematic comparisons of these methods based on relevant performance criteria. Here, we present OPAL, a software package implementing commonly used performance metrics, including those of the first challenge of the Initiative for the Critical Assessment of Metagenome Interpretation (CAMI), together with convenient visualizations. In addition, OPAL implements diversity metrics from microbial ecology, as well as run time and memory efficiency measurements. By allowing users to customize the relative importance of metrics, OPAL facilitates in-depth performance comparisons, as well as the development of new methods and data analysis workflows. To demonstrate the application, we compared seven profilers on benchmark datasets of the first and second CAMI challenges using all metrics and performance measurements available in OPAL. The software is implemented in Python 3 and available under the Apache 2.0 license on GitHub (https://github.com/CAMI-challenge/OPAL).\n\nAuthor summaryThere are many computational approaches for inferring the presence and relative abundance of taxa (i.e. taxonomic profiling) from shotgun metagenome samples of microbial communities, making systematic performance evaluations a very important task. However, there has yet to be introduced a computational framework in which profiler performances can be compared. This delays method development and applied studies, as researchers need to implement their own custom evaluation frameworks. Here, we present OPAL, a software package that facilitates standardized comparisons of taxonomic metagenome profilers. It implements a variety of performance metrics frequently employed in microbiome research, including runtime and memory usage, and generates comparison reports and visualizations. OPAL thus facilitates and accelerates benchmarking of taxonomic profiling techniques on ground truth data. This enables researchers to arrive at informed decisions about which computational techniques to use for specific datasets and research questions.

Source connections

Explore related subjects

bioinformaticsPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Meyer, F., Bremges, A., Belmann, P., Janssen, S., McHardy, A. C., Koslicki, D.. 2018-07-19. Assessing taxonomic metagenome profilers with OPAL. https://doi.org/10.1101/372680

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

The Almond Expression Atlas: A comprehensive multi-tissue transcriptomic resource for almond gene expression

Background: Improving crop adaptation and enhancing agronomic and quality traits requires a deeper understanding of gene expression regulation, for which expression atlases are valuable tools. Almond (Prunus dulcis (Mill.) D.A. Webb) is one of the most economically important tree nut crops worldwide. However, an integrated transcriptomic resource for this species is still lacking. Available data remain fragmented across independent studies, limiting their comparability and reuse. Results: Here, we present the first almond gene expression atlas: a comprehensive multi-tissue transcriptomic resource integrating publicly available RNA sequencing data. A total of 142 high-quality untreated and control samples from 17 independent projects were curated through a metadata standardization process and analyzed using a unified bioinformatics pipeline. Standardized expression profiles were generated for 25,225 annotated genes in the Prunus dulcis Texas v2.0 reference genome. The atlas integrates two complementary expression approaches: a median TPM approach capturing quantitative expression levels and a rank-based strategy implemented using the Bgee expression calling method, designed to improve comparability across heterogeneous datasets. This design enables the identification of tissue-specific candidate genes and the exploration of expression patterns across tissues and developmental stages. Its applicability is demonstrated through two use cases, highlighting its ability to identify biologically meaningful gene sets associated with bud dormancy transitions and to prioritize candidate reference genes for RT-qPCR normalization in almond. Conclusions: The Almond Expression Atlas v1.0 is publicly available through FruitDBase (fruitdbase.csic.es), providing an interactive platform for gene expression exploration and visualization. This resource supports functional genomics studies and facilitates candidate gene discovery in almond, while ensuring data traceability and promoting data reusability.

bioinformatics↗
bioRxiv2026

Leakage-Controlled Benchmarking of Variant Effect Predictors and Development of Gradient-Boosting Consensus Model

Single amino acid mutations are among the most common forms of genetic variation, with effects ranging from negligible to severe disruption of protein function. Accurate quantitative evaluation of protein variants therefore represents a key challenge in medicine and protein engineering. Computational variant effect predictors (VEPs) are widely used to prioritize missense variants, yet their fair comparison is hindered by data circularity, benchmarks restricted to human clinical annotations, and limited availability of code and training data, all of which can inflate apparent performance. Here, we reviewed 18 recent and 3 older, widely adopted variant effect predictors for reproducible benchmarking and assembled a rigorously curated, leakage-controlled Benchmark dataset by integrating clinically annotated human variants from ClinVar and Humsavar as well as cross-species variants with functional annotations. The evaluation on this dataset revealed a narrow overall performance range for the benchmarked tools (AUC 0.631-0.758) and a consistent drop from clinical to functional data. Surprisingly, several tools published two decades ago demonstrated comparable performance to mid-ranked modern predictors. Building on these results, we developed a gradient-boosting consensus model, PredictSNPprot, which achieved an AUC of 0.82 on a held-out test set, outperforming the original PredictSNP meta-predictor and the best individual state-of-the-art tool by 13.6 and 5.7 percentage points, respectively. The Benchmark dataset, including the predictions from the evaluated tools, is available at https://doi.org/10.5281/zenodo.23597130 and PredictSNPprot consensus predictor is available at https://github.com/loschmidt/PredictSNPprot.

bioinformatics↗
bioRxiv2026

Cross-modal transfer enables zero-shot prediction of drug responses across cell types

Genetic perturbation responses capture the functional consequences of gene-level interventions, providing functional information for drug-response prediction beyond baseline gene expression. Within the same cellular context, genetic and chemical perturbations engage shared regulatory networks, providing a biological basis for transferring response information between perturbation modalities. Here we introduce GRAFT, a framework that combines genetic-to-chemical transfer with cross-context prediction to infer drug-induced transcriptional responses without drug measurements in the target context. Across benchmarked LINCS and external contexts, GRAFT improved response correlation, directional agreement and magnitude reconstruction over competing methods, particularly for drugs with context-dependent responses. GRAFT-Adapt further refines predictions of unmeasured responses using a small number of drug measurements from the target context. GRAFT-Atlas spans 13 cellular contexts and 7,856 drug catalog entries, comprising 102,128 predictions for drug-context combinations. The atlas captured recognizable pharmacological neighborhoods, shared functional programs, context-dependent transcriptional changes and structural-functional associations supported by molecular docking. Across ten iPSC lines from distinct donors, GRAFT identified candidate inter-donor differences, with functional support for selected candidates from measured genetic perturbation responses. Together, GRAFT enables perturbation-informed prediction and comparison of drug responses across cellular contexts.

bioinformatics↗