Search bioRxiv⌕ Search

bioRxiv · 10.64898/2026.01.27.702015

Functional In-Context Learning in Genomic Language Models with Nucleotide-Level Supervision and Genome Compression

Abstract

AO_SCPLOWBSTRACTC_SCPLOWGenomic foundation models aim to learn general-purpose representations directly from DNA sequence, enabling sequence understanding, generation, and probabilistic reasoning across a wide range of biological tasks. Scaling such models to genomic lengths, however, remains challenging due to the tension between long-range context, nucleotide-level resolution, and practical computational efficiency. Architectural innovations have enabled increasingly long nominal inputs, but often struggle to translate additional context into meaningful performance gains, particularly in the presence of sparse functional signal along eukaryotic genomes. In this work, we revisit the design of long-context genomic foundation models from the perspective of training objective and data construction. We introduce Factorized Nucleotide Supervision (FNS), which reconciles efficient k-mer tokenization with single-nucleotide likelihoods through probability marginalization, and Genome Compression Pretraining (GCP), which reshapes the training distribution by concentrating on gene-centric and regulatory regions. Together, these techniques enable standard transformer-based models to perform functional in-context learning without sacrificing nucleotide-level fidelity or computational efficiency. Building on these ideas, we present a family of autoregressive genomic foundation models supporting contexts of up to 98k base pairs across eukaryotic and prokaryotic genomes. Across training-free evaluations and downstream fine-tuning benchmarks, our models consistently improve over prior approaches and match or exceed state-of-the-art baselines while enabling substantially more efficient inference. Together, these results demonstrate that aligning supervision and data regimes with the biological structure of genomic sequence provides a principled and effective path toward scalable and biologically faithful genomic language modeling. Models, data, and scripts for downstream analyses are publicly available at https://huggingface.co/GenerTeam.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Li, Q., Zhan, Z., Feng, S., Zhu, Y., He, Y., Wu, W., Shi, Z., Wang, S., Hu, Z., Yang, Z., Li, J., Tang, J., Liu, H., Qin, T.. 2026-01-29. Functional In-Context Learning in Genomic Language Models with Nucleotide-Level Supervision and Genome Compression. https://doi.org/10.64898/2026.01.27.702015

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

Improved long-read sequencing technology reveals full genome architecture of Malian Plasmodium falciparum field isolates

Understanding genome plasticity in malaria parasites and the diversity of their repetitive gene families requires high-quality reference genomes derived from parasites recently isolated from patients. However, generating such genomes in Plasmodium falciparum remains challenging owing to the parasite's extreme AT content, extensive repetitive regions, and the frequent occurrence of multiclonal infections in endemic settings. As a result, most available reference genomes are derived from laboratory-adapted strains, that have experienced prolonged in vitro maintenance, and that incompletely represent natural parasite diversity. We developed and optimized a de novo assembly strategy based on PacBio HiFi long-read sequencing and applied it to eight recently-isolated and cloned P. falciparum parasites from Mali. HiFi sequencing enabled highly accurate genome reconstruction, yielding assemblies with a mean of 18 contigs, 301 pseudogenes, and a frameshift rate of 0.04% following Liftoff annotation. Assemblies showed high structural integrity and near-complete chromosomes, including improved recovery of subtelomeric regions and multigene families, without requiring short-read correction. Despite overall conservation of chromosome organization, substantial variation was observed in chromosome-end architecture, subtelomeric structure, and telomeric repeat content among parasite genomes. Our approach simplifies the generation of high-quality P. falciparum genomes and reduces sequencing requirements fourfold. The resulting assemblies enabled characterization of natural variation in telomere plasticity and multicopy gene family diversity in field isolates. These findings establish PacBio HiFi sequencing as an effective and scalable strategy for producing highly contiguous and accurate de novo assemblies of P. falciparum, supporting the development of geographically-diverse, field-relevant malaria parasite reference genomes.

genomics↗

Cell atlases of schistosome human infective stages reveal larval tissue fates and innovations

Schistosomes infect over 250 million people, yet the cellular basis of their human-infective cercaria larva, that drives transmission and infection, remains poorly understood. We present single-cell atlases of Schistosoma mansoni cercariae, widely regarded as transcriptionally quiescent, and schistosomula, the first intra-mammalian stage. Twenty-two cercarial cell types either persist into schistosomula or are lost. The tail harbours several cercaria-specific innovations, including a novel tegument. Gene-age analysis supports the cercaria as the most recent addition to the life cycle and tail cells as younger than body cells. Persisting cell types undergo clear transcriptional transitions which give new insights into the dynamics of cercarial transformation into schistosomula. These findings link the recent origin of cercarial cell types to the evolution of active dispersal and transmission in digenean parasites.

genomics↗

A genome-guided multilocus sequence analysis scheme to detect and resolve diversity within the X-disease phytoplasma group (16SrIII)

Phytoplasmas of the X-disease group (16SrIII) cause economically important diseases of vegetables, forage and staple crops, fruit trees and ornamentals in South America, where they are diagnosed and classified only by 16S rRNA gene RFLP. Because this gene is highly conserved, strains that differ in host range, vectors and regulatory relevance cannot be distinguished. We developed a multilocus sequence analysis (MLSA) scheme for typing 16SrIII phytoplasmas, using genome data to select the markers and to define reference lineages. Five single-copy loci (tuf, secA, secY, mutM, cimH) were chosen from 16 curated genomes; primers matched all genomes in silico, and four of the five pairs did not amplify phytoplasmas of other groups co-occurring in the region (16SrI, 16SrVII, 16SrXIII). Genome comparisons resolved four lineages, including one restricted to South America that shared only 90.8-93.9% average nucleotide identity with the other lineages, despite > 98.8% 16S rRNA identity. The concatenated MLSA (2,870 nt) reproduced the genome-based lineages and assigned five Argentinian field isolates from weeds, fodder beet, cassava, plum and a native shrub. Identical 16SrIII-B RFLP patterns occurred in different lineages, and one 16SrIII-B isolate grouped within the 16SrIII-J clade, showing that subgroups are not reliable proxies for lineages. The South American lineage had the broadest host range, spanning six plant families. We also detected chimeric rRNA copies and sequencing artefacts that can cause misclassification. This MLSA provides a PCR-based tool to identify 16SrIII lineages in crops, weeds and insects, supporting epidemiological studies, disease management and phytosanitary decisions.

genomics↗