Search bioRxiv⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.64898/2026.08.13.744701

Assessing the Reliability of LLM-Generated Phenotype-Genotype Associations Through External Validation

Sun, C.·Xin, Y.·Zeng, S.·Sunthankar, S. D.·Su, W.-C.·Lynn, J.·Mundo, S.·Babanejad, M.·Feng, Q.·Wei, W.-Q.
View original source ↗Export metadata ↓

Abstract

BackgroundPhenotype-genotype associations underpin precision medicine by enabling disease prevention, early diagnosis, risk stratification, therapeutic target discovery, and personalized treatment. However, the rapid growth of scientific evidence has made manual curation of these associations increasingly labor-intensive, time-consuming, and incomplete. Large Language Models (LLMs) offer a potential path to scalable genomic generation and synthesis of this knowledge, but their ability to accurately identify phenotype-genotype associations and the extent to which these outputs are supported by established genomic knowledge bases remain unclear. Materials and MethodsFour LLMs, Claude Sonnet 4.6, DeepSeek V4 Flash, Gemini 3 Flash Preview, and GPT-5.5, were benchmarked on six zero-shot task categories covering forward and reverse phenotype-gene and phenotype-SNP generation. A total of 4,196 associations were identified from curated inputs and evaluated through a multistage external verification pipeline comprising phenotype normalization, ontology mapping, genomic identifier validation against Ensembl, and evidence verification using both the GWAS Catalog and OMIM. Associations were assigned a fused evidence level of strong, moderate, weak, or none. ResultsOverall, 74.19% of generated associations were matched to at least one external genomic knowledge base; 9.15% received strong support and 54.46% moderate support. Phenotype-gene associations were more verifiable than phenotype-SNP associations (strong or moderate: 67.19% vs 54.06%). Among existing associations, Claude Sonnet 4.6 achieved the highest overall strong or moderate rate (69.2%), followed by GPT-5.5 (65.1%), DeepSeek V4 Flash (61.7%), and Gemini 3 Flash Preview (56.9%). ConclusionLLMs can support scalable generation of candidate phenotype-genotype associations. Performance varied substantially by relation type and was lower for SNP-level and rare disease associations, highlighting both the limitations of current genomic resources and the need for rigorous validation pipelines.

Explore related subjects

bioinformaticsPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Sun, C., Xin, Y., Zeng, S., Sunthankar, S. D., Su, W.-C., Lynn, J., Mundo, S., Babanejad, M., Feng, Q., Wei, W.-Q.. 2026-08-21. Assessing the Reliability of LLM-Generated Phenotype-Genotype Associations Through External Validation. https://doi.org/10.64898/2026.08.13.744701

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

CellChat Hotspot: A Targeted Approach to Cell Cell Communication Analysis With Spatial transcriptomics Validation.

Abstract. In recent years, single cell RNA sequencing (scRNA seq) has enabled the investigation of intercellular communication through Cell Cell Communication (CCC) tools such as CellChat. Despite their potential, these approaches remain challenging to interpret, particularly when global analyses generate complex interaction networks that may include biologically irrelevant signals. To address this limitation, we developed a framework termed CellChat Hotspot, designed to improve the biological interpretability of inferred communication networks. This approach integrates gene set enrichment analysis with a context driven filtering strategy, selecting both genes and cells based on their functional relevance prior to communication inference. Applied to stage II colon cancer data, CellChat Hotspot enabled the identification of condition-specific interaction patterns associated with recurrence. Non recurrent patients showed enrichment of immune related processes, whereas recurrent cases were characterized by pathways associated with invasion and metastasis. Notably, a loss of interaction in the HSPG pathway between CALB1+ and stromal cells was observed in recurrent patients, suggesting a potential disruption of cell matrix adhesion mechanisms. To verify the reliability of this finding, we applied the framework to spatial transcriptomics data, which confirmed the presence of the HSPG2 DAG1 interaction in non-recurrent tumors and its near absence in recurrent ones. Overall, this framework provides a biologically guided strategy for studying intercellular communication in complex systems such as the tumor microenvironment.

bioinformatics↗
bioRxiv2026

SIFT: Syncmer-Informed Selection of Haplotypes for T2T Imputation

Genome patching improves the completeness of fragmented assemblies by filling unresolved gaps with sequences from reference genomes. Pangenome panels provide diverse candidate haplotypes for this task, but identifying suitable references efficiently becomes increasingly challenging as these panels grow. We present SIFT, a compressed indexing method for selecting reference haplotypes for genome patching with ImpuT2T. SIFT combines closed-syncmer parsing, phrase-level indexing, and pseudo-maximal exact match (pseudo-MEM) queries. Closed syncmers provide a context-independent parsing rule with bounded distances between consecutive selected positions, enabling consistent parsing of reference and query sequences. We demonstrate that two closed syncmers determine the distance at which they can occur consecutively in any string, so that equal substrings of two parses represent identical sequences and no gap sequence need be stored. SIFT indexes the reference parse and its reverse using run-length compressed suffix arrays and adapts the Boyer-Moore-Li search procedure to identify pseudo-MEMs between the reference parse and query parse. It then aggregates matching evidence to identify candidate reference haplotypes for downstream patching. This framework connects compressed sequence indexing with haplotype selection and provides a basis for querying large reference panels at the phrase level. SIFT is publicly available under the GNU license at https://github.com/EddieFerro/SIFT.

bioinformatics↗
bioRxiv2026

Assessing the Capacity of Genomic Language Models to Generate Informative Embeddings of Viral Genome Fragments

Genomic language models (gLMs) trained on large volumes of nucleotide sequences are becoming an alternative to classical methods in metagenomic analysis, particularly for studying genomic "dark matter". In this work, we investigate the ability of four gLMs (DNABERT-2, DNABERT-S, ViraLM, and Evo2-7B) to generate taxonomically informative vector representations of short viral genome fragments. We constructed a balanced reference dataset of viruses based on the ICTV Virus Metadata Resource and compared the representations generated by gLMs with conventional k-mer profiles and MinHash sketches using clustering and neighborhood-based metrics (adjusted Rand index (ARI), precision@k, and CROSS precision@k). On the reference dataset, three of the four gLMs outperformed the 4-mer profile in terms of ARI and precision@5 under the same evaluation protocol. The best performance was generally achieved by intermediate model layers. Conventional k-mer profiles consistently outperformed MinHash sketches. We further evaluated the methods on two viral orders (egoviruses and mriyaviruses) absent from the reference dataset, with model layers and k-mer lengths selected exclusively on the reference dataset. DNABERT-S showed the most consistently strong performance across these two orders, ranking first on mriyaviruses and second on egoviruses, while k-mer profiles with Jensen-Shannon distance remained competitive. We also observed the consistency of representations between forward fragments and their reverse complements. Overall, the best-performing gLMs surpassed conventional baselines, although their advantage depended on the dataset and evaluation metric. The consistently strong performance of DNABERT-S suggests a potential benefit of contrastive fine-tuning and indicates that larger models are not necessarily more effective for generating taxonomically informative representations.

bioinformatics↗