Biology subjects

Oktarina, A.

Publications and source records attributed to Oktarina, A..

2 recordsLinked to original sources

bioRxiv2025

Evaluating Large Language Models for Gene-to-Phenotype Mapping: The Critical Role of Full-Text Database Access

Transformer-based large language models (LLMs) have demonstrated significant potential in the biological and medical fields due to their ability to effectively learn from large-scale, diverse datasets and perform a wide range of downstream tasks. However, LLMs are limited by issues such as information processing inaccuracies and data confabulation. These limitations hinder their utility for literature searches and other tasks requiring accurate and comprehensive extraction of information from extensive scientific literature. In this study, we evaluated the performance of various LLMs in accurately retrieving peer-reviewed literature and mapping correlations between 198 genes and six phenotypes: bone formation, cartilage formation, fibrosis, cell proliferation, tendon formation, and ligament formation. Our analysis included three types of models. First, standard transformer-based LLMs (ChatGPT-4o and Gemini 1.5 Pro). Second, specialized LLMs with dedicated custom databases containing peer-reviewed articles (SciSpace and ScholarAI). Third, specialized LLMs without dedicated databases (PubMedGPT and ScholarGPT). Using human-curated gene-to-phenotype mappings as the ground truth, we found that specialized LLMs with dedicated databases achieved the highest accuracy (>80%) in gene-to-phenotype mapping. Additionally, these models were able to provide relevant peer-reviewed publications supporting each gene-to-phenotype correlation. These findings underscore the importance of database augmentation and specialization in enhancing the reliability and utility of LLMs for biomedical research applications.

genetics↗

bioRxiv2025

Database-Augmented Transformer-Based Large Language Models Achieve High Accuracy in Mapping Gene-Phenotype Relationships

Transformer-based large language models (LLMs) have demonstrated significant potential in the biological and medical fields due to their ability to effectively learn from large-scale, diverse datasets and perform a wide range of downstream tasks. However, LLMs are limited by issues such as information processing inaccuracies and data confabulation, which hinder their utility for literature searches and other tasks requiring accurate and comprehensive extraction of information from extensive scientific literature. In this study, we evaluated the performance of various LLMs in accurately retrieving peer-reviewed literature and mapping correlations between 102 genes and four phenotypes: bone formation, cartilage formation, fibrosis, and cell proliferation. Our analysis included standard transformer-based LLMs (ChatGPT4o and Gemini1.5 Pro), fine-tuned LLMs with dedicated custom databases containing peer-reviewed articles (SciSpace and ScholarAI), and fine-tuned LLMs without dedicated databases (PubMedGPT and ScholarGPT). Using human-curated gene-to-phenotype mappings as the ground truth, we found that fine-tuned LLMs with dedicated databases (SciSpace and ScholarAI) achieved high accuracy (>80%) in gene-to-phenotype mapping. Additionally, these models were able to provide relevant peer-reviewed publications supporting each gene-to-phenotype correlation. These findings underscore the importance of database augmentation and finetuning in enhancing the reliability and utility of LLMs for biomedical research applications.

genomics↗