Lain, A. D. | Search bioRxiv

bioRxiv2025

High-precision Biomedical Text Corpora for Multi-Entity Recognition: A CoDiet study

We present here five biomedical, multi-entity corpora that can be used as benchmarks for named-entity recognition (NER), targeted to literature on metabolic syndrome. The CoDiet-Gold corpus contains annotations for 500 full-text publications and 348,406 annotations. It is divided into CoDiet-Gold-public (450 documents) and CoDiet-Gold-private (50 documents). Each document was independently annotated by two human experts, with disagreements fully adjudicated by a third expert. The CoDiet-Electrum corpus (2,998,273 annotations) contains 4,423 publications that were annotated using case-insensitive matching of the surface forms with punctuation ignored, found in CoDiet-Gold-public. Finally, for the same 4,423 documents, two fully machine annotated corpora CoDiet-Bronze (2,938,738 annotations) and CoDiet-Silver (2,298,988 annotations), were created by utilising existing NER algorithms to annotate these. These corpora contain categories (organisms, disease, genes, proteins, metabolites) that add depth to existing corpora, as well as new categories that do not appear in other corpora (food, dietary methods, sample types, computational methods, study methodology, population characteristics, data types, and microbiome).

bioinformatics↗

bioRxiv2025

Microbial Named Entity Recognition and Normalisation for AI-assisted Literature Review and Meta-Analysis

MotivationManual curation of biomedical literature is slow and error-prone and while large language models (LLMs) trained on general texts have shown to be useful for text summarisation, these methods lack the domain-specific expertise required to perform this task accurately. Here we describe the creation of the first microbiome-specific text corpus, use this to train deep learning algorithms for named-entity recognition (NER) and normalisation (NEN), and demonstrate their use to meta-analyse microbiome literature. MethodsWe developed an automated pipeline to annotate all mentions of bacteria, archaea, and fungi in 1,410 full-text microbiome articles. We manually annotated (gold-standard) a separate test set of 288 documents. We trained different transformer-based language models for microbiome recognition and normalisation to taxonomic identifiers and evaluate their performance using the precision, recall, F1-score, and accuracy on the test set. The best models were used to automatically annotate all available Open Access, full-text microbiome articles (n=6,927) and identify taxa that are significantly overrepresented across 14 domains. ResultsThe training and validation set contained a total of 90,150 annotations (both long form and abbreviations). Using the gold-standard test set, with an inter-annotator agreement rate of 99.52% for NER and 88.31% for NEN, the trained models were evaluated and our fine-tuned BioBERT model achieved an F1-score of 96% for NER surpassing a rule- and dictionary-based annotation pipeline (94%). For NEN the accuracy obtained by the deep learning models greatly surpassed that of the pipeline (91% vs 69%). Evaluated across the entire available literature, our models annotate an entire full-text document in only 7 seconds. ConclusionOur algorithms have near perfect precision and greatly speed up the process of annotating microbes in full-text articles. We demonstrated the capabilities of these methods by analysing the entire available literature and describe the taxa associated with each of the domains in our meta-analysis, and exemplify how these methods can be integrated into literature review workflows improving both the speed and accuracy of results. AvailabilityAll codes and data for automatic annotation, model training, and generation of taxonomic trees visualising the data will be made available following peer review with instructions on how to deploy the model on new texts from https://github.com/omicsNLP/microbELP.

bioinformatics↗