Search bioRxiv⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.64898/2026.02.02.703356

Embarrassingly_FASTA: Enabling Recomputable, Population-Scale Pangenomics by Reducing Commercial Genome Processing Costs from $100 to less than $1

Walsh, D. J.·Njie, e. G.
View original source ↗Export metadata ↓

Abstract

Computational preprocessing has become the dominant bottleneck in genomics, frequently exceeding sequencing costs and constraining population-scale analysis, even as large repositories grow from tens of petabytes toward exabyte-scale storage to support World Genome Models. Legacy CPU-based workflows require many hours to days per 30x human genome, driving many repositories to distribute aligned or derived intermediates such as BAM and VCF files rather than raw FASTQ data. These intermediates embed reference- and model-dependent assumptions that limit reproducibility and impede reanalysis as reference genomes, including pangenomes, continue to evolve. Although recent work has established that GPUs can dramatically accelerate genomic pipelines, enabling large-cohort processing to shrink from years to days given sufficient parallelism, such workflows remain cost-prohibitive. Here, we introduce Embarrassingly_FASTA, a GPU-accelerated preprocessing pipeline built on NVIDIA Parabricks that fundamentally changes the economics of genomic data management. By rendering intermediate files transient rather than archival, Embarrassingly_FASTA enables retention of raw FASTQ data and reliable use of highly discounted ephemeral cloud infrastructure such as spot instances, reducing compute spend from [~]$17/genome (CPU on-demand) to <$1/genome (GPU spot), and commercial secondary-analysis pricing from [~]$120/genome to compute spend under $1/genome. We demonstrate the impact of this efficiency using a simulated large-cohort pangenome build-up (using variant-union accumulation as a proxy for diversity growth) in Caenorhabditis elegans and humans, highlighting the long tail of unsampled human genetic diversity. Beyond GPU kernels, Embarrassingly_FASTA contributes a transient-intermediate lifecycle and spot-friendly orchestration that makes FASTQ retention and routine recomputation economically viable. Embarrassingly_FASTA thus provides enabling infrastructure for recomputable, population-scale pangenomics and next-generation genomic models. Non-Expert DescriptionReading a persons complete DNA sequence has become fast and inexpensive, but turning that raw data into something scientists can analyze is now one of the biggest obstacles in modern genetics. Today, processing a single genome can take many hours or even days, which makes it difficult and expensive to study large populations or reanalyze data when better methods become available. As a result, many databases store only partially processed DNA instead of the original data, limiting future discoveries. In this work, we present a new system that dramatically speeds up this processing step using graphics processing units (GPUs), the same type of hardware used in modern artificial intelligence. With our approach, a human genome can be processed in about 35 minutes instead of more than 15 hours, and at a fraction of the cost. This makes it practical to keep the original DNA data and reprocess it whenever new tools or reference genomes become available, rather than being locked into outdated results. We also show that this speed and affordability allow researchers to explore genetic diversity at an unprecedented scale. By analyzing both human genomes and those of a small worm species commonly used in research, we demonstrate how new genetic variations continue to emerge as more individuals are studied, especially in humans, where much diversity remains unexplored. Overall, our work helps remove a major barrier to studying DNA at population scale and lays the foundation for future genetic models that could better explain disease, evolution, and human health. O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=112 SRC="FIGDIR/small/703356v2_ufig1.gif" ALT="Figure 1"> View larger version (69K): org.highwire.dtl.DTLVardef@11b8fc3org.highwire.dtl.DTLVardef@7b8b60org.highwire.dtl.DTLVardef@fb5a3borg.highwire.dtl.DTLVardef@1e11276_HPS_FORMAT_FIGEXP M_FIG C_FIG

Explore related subjects

bioinformaticsPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Walsh, D. J., Njie, e. G.. 2026-02-04. Embarrassingly_FASTA: Enabling Recomputable, Population-Scale Pangenomics by Reducing Commercial Genome Processing Costs from $100 to less than $1. https://doi.org/10.64898/2026.02.02.703356

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

deepTools4: Faster, Scalable, and Modernized Genomic Data Analysis

The deepTools suite is widely used for quality control, summarization, and visualization of high-throughput sequencing data. However, as genomic datasets grow in scale and depth, existing tools often become computational bottlenecks. We present deepTools4, a modernized framework designed for high performance, scalability, and enhanced usability, while maintaining seamless integration with the Galaxy platform. Core functionality has been rewritten and thoroughly benchmarked, achieving substantial computational speed-ups. Visualization tools have been modernized to provide more intuitive, high-quality graphical outputs. In addition, our testing framework has been expanded to facilitate continuous development and ensure robustness. We demonstrate the utility of deepTools4 across diverse epigenomics and transcriptomics datasets, highlighting its ability to summarize genome-wide signals, apply a wide range of common normalization methods, and visualize enrichment patterns. The enhanced deepTools4 framework delivers a faster, more scalable solution for reproducible genome-wide analysis and interpretation across many sequencing modalities. Availability: https://github.com/deeptools/deepTools/

bioinformatics↗
bioRxiv2026

Reproducibility and cross-dataset concordance of drug-induced transcriptional responses

Drug-perturbation transcriptomic resources are expanding, but the reproducibility and cross-source consistency of the responses they measure remain critical to their use. We compared six resources using independent replicates, matched-drug analyses, and response decomposition. High agreement in treated expression did not translate into equally repeatable drug-induced changes. Cross-source response concordance was generally limited, including under matched cell and treatment conditions, although some comparisons retained drug-specific information. In Tahoe, a control-expression-only baseline left substantial drug-specific variation unresolved. Measured drug main effects - the average responses across cell lines - provided information for predicting cell-specific interactions, with this relationship retained in a cross-plate validation. Exploratory analyses of structure, target, and external-response representations identified modest signals in selected pipelines. Together, these analyses characterize the reliability and transferability of current perturbation resources and identify prediction of drug main effects as a key bottleneck in the framework examined, motivating shared reference measurements and better recovery of drug-specific responses.

bioinformatics↗
bioRxiv2026

Propensity-Adjusted AUC under Nuisance-Model Misspecification

Propensity adjustment compares medical-image detector scores within groups with similar measured properties, such as stain or focus. We examine its sensitivity to the nuisance model that forms those groups. Under Gaussian latent-index assumptions and exact linear-index strata, a closed-form correlation gives an exact boundary for adjusted AUC to fall below one half. Quadrature matches 36 simulation cells within 0.008. Inflation and attenuation relative to correct adjustment are empirical findings, not general ordering results. An overlap screen and a self-stratified control introduced for this study miss 140 of 141 simulated cells with absolute error above 0.05: checking the fitted propensity cannot certify completeness of the adjustment set. On real pathology corpora, logistic and boosted-tree models using the same 16 features disagree. Among splits where both pass both checks, mean adjusted-AUC gaps are 0.057 on acid-fast bacilli (AFB), 0.048 on Kaggle, and 0.049 on PCam, over three, five, and three splits respectively. These gaps measure model sensitivity; neither estimate is known to be correct. A controlled colour shift moves adjusted AUC relative to model-specific references in the predicted direction in 78 of 80 label-dependent cells. Broader balance and label-fit diagnostics favour trees without certifying their adjusted AUC. We recommend reporting sensitivity across nuisance models alongside these diagnostics. Reversal relative to known truth occurs in constructed stress tests; its frequency in practice remains unknown.

bioinformatics↗