Biology subjects

Zou, J.

Publications and source records attributed to Zou, J..

At least 55 records · Page 3Linked to original sources

bioRxiv2025

VSA-2, a novel plant-derived adjuvant for SARS-CoV-2 subunit vaccine

QS-21, a key component of several licensed vaccines is facing limited supply, dose-limiting toxicity and other drawbacks which together limit its broader usage. Development of saponin alternatives to QS-21 that retain its desirable adjuvant activity without its drawbacks is in high need. Incorporating an amide side chain into the more sustainable Momordica saponins (MS) I and II led to the recent discovery of two semisynthetic immunostimulatory adjuvants VSA-1 and VSA-2. Here, we showed that the receptor-binding protein (RBD) of ancestral SARS-CoV-2 adjuvanted with VSA-2 (VSA-2-RBD) induced high titers of SARS-CoV-2-specific humoral and T helper-1 prone immune responses in mice comparable to that triggered by QS-21-RBD. Vaccination with VSA-2-RBD provided strong protection against SARS-COV-2, Delta and Omicron variants infection and the virus-induced lung inflammation and pathology similarly as QS-21-RBD vaccination. Overall, our results suggest that VSA-2 adjuvant can potentially complement the clinically proven saponin adjuvant QS-21 in vaccines against infectious diseases.

microbiology↗

bioRxiv2025

RobPicker: A Meta Learning Framework for Robust Identification of Macromolecules in Cryo-Electron Tomograms

Accurate particle picking of macromolecules of all kinds, shapes, and sizes in cryogenic electron tomograms (cryo-ET) is critical for understanding the molecular architecture of biological systems in their native state. Current deep learning methods have shown potential in identifying macromolecules from tomograms, but they are vulnerable to issues of noise in the training dataset obtained through human or automatic labeling and imbalanced distribution of macromolecule species. To address these limitations, we developed RobPicker, a meta-learning framework that effectively mitigates these issues by automatically learning deep neural networks to correct label errors and give greater emphasis to underrepresented macromolecule species. In evaluations across diverse cryo-ET datasets with noisy labels and imbalanced species distributions, RobPicker substantially outperforms state-of-the-art methods, particularly in identifying small and rare macromolecules. The efficiency and robustness of RobPicker can also be used for rapid fine-tuning of the tilt-series alignment, leading to improved tomogram reconstruction and enabling high-resolution cellular structural biology analysis.

bioinformatics↗

bioRxiv2025

scSTAR2: a multiomics integration algorithm to reveal disease specific cellular signatures by bridging single-cell resolution features and clinical metadata

Single-cell techniques, pivotal in characterizing intricate cell and spatial structures within tissues, face challenges in contextualizing clinical phenotypes. Currently, most investigations of the clinical phenotype related cell/spatial heterogeneities relied on the phenotype information from single-cell data. However, a wealth of underutilized clinical metadata exists within conventional bulk sequencing data. Current methods for correlating clinical information with individual cells or spatial regions remain limited and lack robustness. Here we present scSTAR2, a novel algorithm that transcends existing limitations by reconstructively integrating multiomics data to identify cells associated with clinical phenotypes. Unlike existing methods, scSTAR2 reconstructs single-cell data guided by specific phenotypes, significantly reducing interference from irrelevant noise. By employing scSTAR2 to integrate scRNA-seq, scATAC-seq, and spatial transcriptomics and bulk RNA-seq, we not only confirmed a new heat-shock Treg subtype in tumors but also more sensitively identified TLS (tertiary lymphoid structure) areas than traditional methods. In conclusion, scSTAR2 has proven to significantly enhance single-cell data interpretation across diverse clinical scenarios.

bioinformatics↗

bioRxiv2025

DANCE 2.0: Transforming single-cell analysis from black box to transparent workflow

Preprocessing is a critical step in single-cell data analysis, yet current practices remain largely a black-box, trial-and-error process driven by user intuition, legacy defaults, and ad hoc heuristics. The optimal combination of steps such as normalization, gene selection, and dimensionality reduction varies across tasks, model architectures, and dataset characteristics, hindering reproducibility and method development. We present DANCE 2.0, an automated and interpretable preprocessing platform featuring two key modules: the Method-Aware Preprocessing (MAP) module, which discovers optimal pipelines for task-specific methods via hierarchical search, and the Dataset-Aware Preprocessing (DAP) module, which recommends pipelines for new datasets via similarity-based matching to a reference atlas. Together, MAP and DAP execute over 325,000 pipeline searches across six major tasks - clustering, cell type annotation, imputation, joint embedding, spatial domain identification, and cell type deconvolution - yielding robust and generalizable recommendations. MAP-recommended pipelines consistently outperform original method defaults, with substantial gains across all tasks. Beyond automation, DANCE 2.0 reveals interpretable preprocessing patterns across tasks, methods, and datasets, transforming preprocessing into a transparent, data-driven process. All resources are openly available at https://github.com/OmicsML/dance to support broad community adoption and future methodological advances.

bioinformatics↗

bioRxiv2025

ODFormer: a Virtual Organoid for Predicting Personalized Therapeutic Responses in Pancreatic Cancer

Pancreatic cancer (PC) patient-derived organoids (PDOs) faithfully recapitulate therapeutic responses but face clinical translation barriers, including high costs and technical complexity. To address these problems and the lack of frameworks for PDO-based drug-response assays, we developed ODFormer, a computational framework that simulates PC PDOs to predict clinically actionable, patient-specific drug responses by integrating transcriptomic and mutational profiles. ODFormer first employed two encoders, pretrained on 30,000 pan-cancer bulk transcriptomics and 1 million PC single-cell profiles respectively, to distil tissue-and organoid-specific representations. Then, trained on our curated 14,000 PDO drug-response assay (across 183 PDOs and 98 drugs) using a transformer-augmented hybrid contrastive network, ODFormer significantly outperformed state-of-the-art methods, notably achieving a PCC >0.9 in predicting standardized drug response. Multi-cohort retrospective analyses further demonstrated that ODFormer-guided personalized therapy significantly improves clinical outcomes, without requiring physical organoid assays. Furthermore, ODFormer identified novel clinico-biological PC subtypes and revealed therapy resistance biomarkers by stratifying predicted responders and non-responders. These were validated using independent datasets including TCGA-PDAC. Notably, ODFormer-guided treatment efficacy showed high concordance with prospective clinical responses by CA19-9.

bioinformatics↗

bioRxiv2025

The Ins and Outs of Manganese: ZIP14 facilitates the efflux of excess manganese from the brain.

Manganese (Mn) is essential for many enzymatic processes in the brain; however, Mn overload can lead to neurotoxicity and behavioral deficits. The blood-brain barrier (BBB), comprised of polarized endothelial cells, tightly regulates metals in and out of the brain. ZIP14 (SLC39A14) is a metal transporter more recently found to transport Mn, with mutations in human SLC39A14 resulting in brain Mn accumulation and neurological deficits. However, ZIP14s precise localization and role in BBB endothelial cells remain unclear. Here, we show in vivo ZIP14 expression in BBB endothelial cells, which upregulates following Mn supplementation. Using expansion microscopy, we observed a shift in ZIP14 localization from an equal apical-basolateral distribution to predominantly basolateral after Mn exposure. Endothelial-specific Zip14 KO (EKO) mice exhibited impaired Mn efflux from the brain and increased brain Mn accumulation after nasal delivery and dietary Mn supplementation. In vitro studies using primary endothelial cells from EKO mice and ZIP14-overexpressing hCMEC/D3 cells confirmed that ZIP14 primarily mediates basolateral-to-apical Mn transport. Collectively, our results demonstrate that ZIP14 is critical for brain Mn clearance, highlighting its potential as a therapeutic target to mitigate Mn-induced neurotoxicity. Significance StatementOur study reveals that the endothelial metal transporter ZIP14 plays a critical role in removing excess Mn from the brain. Using expansion microscopy, genetic manipulation, metallomics, and transport studies employing radiolabeled Mn, we demonstrate that ZIP14 undergoes strategic relocalization in response to Mn exposure and primarily functions in basolateral-to-apical transport. Deletion of endothelial ZIP14 leads to brain Mn accumulation, establishing its fundamental role in protecting against Mn-induced neurotoxicity. This finding provides mechanistic insight into how the brain protects itself against Mn overload and why ZIP14 mutations in humans result in brain Mn accumulation.

physiology↗

bioRxiv2025

Pp6-Pfkfb1 axis modulates intracellular bacterial proliferation by orchestrating host-pathogen metabolic crosstalk

Intracellular bacterial pathogens exhibit heterogeneous replication rates within host macrophages, but the mechanisms by which they manipulate host factors for survival remain incompletely understood. Using a fluorescence-dilution reporter system in Salmonella typhimurium (Salmonella)-infected macrophages, we found that Protein Phosphatase 6 (Pp6) was downregulated in macrophages harboring growing bacteria. Conditional knockout of Pp6 elevated host susceptibility to Salmonella-mediated lethality due to compromised antimicrobial defenses. MicroRNA-31 (miR-31) was identified as a negative regulator of Pp6, and its conditional ablation enhanced bacterial clearance. Yeast two-hybrid screening identified 6-phosphofructo-2-kinase/fructose-2,6-biphosphatase 1 (Pfkfb1), a metabolic regulator, as a substrate of Pp6. Pp6 deficiency resulted in significantly elevated expression of Pfkfb1, which was highly expressed in macrophages containing replicating Salmonella. Pfkfb1 deletion restricted bacterial proliferation by promoting nitric oxide (NO) production while concurrently suppressing arginase-1 (Arg-1) expression and impairing arginine metabolism in macrophages. Collectively, these results establish the Pp6-Pfkfb1 axis as a key regulator of host metabolic adaptation and intracellular bacterial survival, offering potential therapeutic targets against multidrug-resistant pathogens. Author summaryIntracellular pathogens secrete effector proteins that intercept and modify host cells to usurp host defenses and establish habitable intracellular niches, yet the host factors they manipulate for survival and virulence are not fully understood. Using a fluorescence-dilution reporter system in Salmonella typhimurium (Salmonella)-infected macrophages, we found Protein Phosphatase 6 (Pp6) was significantly decreased in macrophages harboring replicating Salmonella and its deficiency indulged bacterial growth in macrophages. However, the deletion of microRNA-31 (miR-31), a negative regulator of Pp6, enhanced Salmonella clearance. Furthermore, through yeast two- hybrid screening, we identified 6-phosphofructo-2-kinase/fructose-2,6-biphosphatase 1 (Pfkfb1) is a substrate of Pp6. And we demonstrated Pfkfb1, abundantly expressed in macrophages containing growing bacteria, is crucial for Salmonella proliferation by regulating nitric oxide (NO) levels, arginase-1 (Arg-1) expression and arginine metabolism. Overall, the molecular changes in Pp6-Pfkfb1 axis drive host metabolic adaptations that enable intracellular bacterial survival.

microbiology↗

bioRxiv2025

UniCure: A Foundation Model for Predicting Personalized Cancer Therapy Response

Predicting drug efficacy across diverse patient contexts remains a major challenge in oncology, as models trained on cancer cell lines often fail to capture patient-specific biology. Emerging biological foundation models and patient-derived technologies offer a promising solution. Here, we present UniCure, the first pre-trained foundation model integrating both omics and chemical foundation models (UCE and Uni-mol) to predict transcriptomic responses to drugs across diverse cellular and tissue contexts, enabling personalized cancer therapy and drug prioritization at the individual level. Rather than encoder/decoder used in traditional models, UniCure utilizes parameter-efficient fine-tuning (PEFT) techniques for optimizing the training process, a novel FlexPert module for modeling flexible drug-cell interactions, and a Maximum Mean Discrepancy (MMD) loss for learning unpaired data. Trained on over 1.8 million perturbation RNA-seq profiles over 22,000 compounds, 166 cell types, and 24 tissues, UniCure achieves high accuracy in predicting both dose-dependent responses and drug combination effects, demonstrating strong generalization across bulk and single-cell transcriptomic data. In particular, fine-tuning on our patient-derived tumor-like clusters and real-world data of 800 profiles enables UniCure to generate individualized therapeutic predictions on patients tissue samples. In addition, UniCure enables patient stratification based on the predicted drug responses, providing a new way for subtyping patients. UniCures drug prioritization was validated across over 1000 patients from pan-cancer cohorts and supported by experiments of candidate therapeutics. By enabling the potential to screen millions of compounds per patient at scale, UniCure represents a biologically grounded tool that could advance personalized precision oncology and accelerate drug discovery.

bioinformatics↗

bioRxiv2025

Predicting Spatial Transcriptomics from H&E Image by Pretrained Contrastive Alignment Learning

The intricate molecular landscape within tissues holds crucial information about cellular behavior and disease progression, yet capturing this complexity at a spatial level remains challenging. While Spatial transcriptomics (ST) offers valuable insights into gene expression patterns within their native tissue context, its widespread adoption is hindered by high costs and limited gene detection capabilities. Here we introduce CarHE (Contrastive Alignment of gene expRession for hematoxylin and eosin image), a method that overcomes these limitations by accurately predicting high-dimensional ST data (over 10,000 genes) solely from readily available H&E (Hematoxylin and Eosin) stained images. This novel pre-trained architecture employs contrastive learning through two mechanisms: cell-type-based transcriptomics information transfer and image-based histology information transfer. These mechanisms precisely align image features with spatial single-cell gene expressions, achieving prediction accuracies exceeding 0.7 (up to 1.7 folds compared to second best) across diverse tissue types and species. CarHEs superior performance extends to identifying subtle pathological features such as tertiary lymphoid structures in various cancers, including breast cancer, lung cancer, melanoma and ccRCC (clear cell renal cell carcinoma), and reconstructing 3D spatial transcriptomics from images alone, offering a cost-effective and robust alternative for large-scale spatial transcriptomics. We further validated CarHEs effectiveness by predicting DFS (Disease-Free Survival) from >1,600 lung cancer patients HE images, achieving a significantly higher AUC (Area Under the Receiver Operating Characteristic Curve) of 0.73 compared to state-of-the-art alternatives (0.58-0.64).

bioinformatics↗

bioRxiv2025

MeCP2 requires interactions with nucleosome linker DNA to read chromatin DNA methylation

Methyl-CpG-binding protein 2 (MeCP2) is an epigenetic reader essential for neuronal function, but how it binds DNA methylation within chromatin is unclear. Using designer nucleosomes we observe that MeCP2 preferentially engages DNA methylation positioned at multiple sites around the nucleosome. Surprisingly, MeCP2 can bind methylated DNA in bent, histone-contacting core nucleosomal DNA. However, this activity requires additional DNA linker interactions, using MeCP2 regions beyond its canonical methyl binding domain. We mapped a novel DNA-binding region in MeCP2 required for this function. Furthermore, histone H1 antagonises the MeCP2-nucleosome interaction by competing for linker DNA. Overall, this reveals that MeCP2 combines nonspecific but essential interactions with linker DNA to aid specific binding to nucleosomal methylated DNA, independent of nucleosome structure. Our findings reveal a novel contribution to full MeCP2 function, and mechanistic insight into how this clinically important protein interacts with chromatin.

biochemistry↗

bioRxiv2025

Spatial multi-omics and deep learning reveal fingerprints of immunotherapy response and resistance in hepatocellular carcinoma

Despite advances in immunotherapy treatment, nonresponse rates remain high, and mechanisms of resistance to checkpoint inhibition remain unclear. To address this gap, we performed spatial transcriptomic and proteomic profiling on human hepatocellular carcinoma tissues collected before and after immunotherapy. We developed an interpretable, multimodal deep learning framework to extract key cellular and molecular signatures from these data. Our graph neural network approach based on spatial proteomic inputs achieved outstanding performance (ROC-AUC > 0.9) in predicting patient treatment response. Key predictive features and associated spatial transcriptomic profiles revealed the multi-omic landscape of immunotherapy response and resistance. One such feature was an interface niche expressing restrictive extracellular matrix factors that physically separates tumor tissue and lymphoid aggregates in nonresponders. We integrate this and other spatially-resolved signatures into SPARC, a multi-omic "fingerprint" comprising scores for immunotherapy response and resistance mechanisms. This study lays groundwork for future patient stratification and treatment strategies in cancer immunotherapy.

cancer biology↗

bioRxiv2025

CellVoyager: AI CompBio Agent Generates New Insights by Autonomously Analyzing Biological Data

Modern biology increasingly relies on complex, high dimensional datasets such as single-cell RNA sequencing (scRNA-seq). However, the richness of such data means that conventional analyses may only scratch its surface. Extracting meaningful insights from these datasets often requires advanced computational methods and domain expertise. Current AI agents for biology are primarily focused on executing user-specified commands and are therefore limited by the users creativity and familiarity with which kinds of analyses are useful. Furthermore, these agents do not account for prior analyses already attempted by researchers, reducing their ability to build upon existing work. To address these limitations, we introduce CellVoyager, an AI agent that autonomously explores scRNA-seq datasets in novel directions conditioned on prior user-ran analyses. Built on large language models, CellVoyager ingests both the dataset and a record of prior analyses to generate and test new hypotheses within a Jupyter notebook environment. We evaluate CellVoyager on CellBench, a new benchmark based on 50 published scRNA-seq studies encompassing 483 analyses. Given only the background sections of these papers, CellVoyager outperformed GPT-4o and o3-mini by up to 20% in predicting which analyses the authors eventually conducted. We then carried out three in-depth case studies where CellVoyager is given previously published papers with their scRNA-seq datasets and conducts analyses to generate new findings. The original authors of each study evaluated these findings and consistently rated them as creative and sound; 80% of the agents hypotheses were deemed scientifically interesting. For example, in one case study, the agent found that CD8+ T cells in COVID-19 infection are more primed for pyroptosis, which was not explored by the original researchers. CellVoyager also reanalyzed a brain aging dataset to discover a previously unreported association between increased transcriptional noise and aging in the subventricular zone of the brain. These results demonstrate that CellVoyager can act both autonomously and collaboratively to accelerate hypothesis generation and computational biology. It also highlights the potential of agents like CellVoyager to unlock new biological insights by reanalyzing the vast existing biological data at scale.

bioinformatics↗

bioRxiv2025

RL-Finetuning of OpenAI o1-mini to Enhance Biomedical Reasoning

Recent breakthroughs in advanced reasoning large language models (LLMs), such as OpenAIs o1, have achieved impressive results in domains like math and coding. However, its not clear how much this type of reasoning helps in solving biomedical problems that involve more domain specialized knowledge and open-ended reasoning. Across two biomedical domains--gene characterization and small molecule property prediction--we find that the commercially available o1-mini model does not consistently outperform non-reasoning LLMs like GPT-4o. This motivated us to explore how much we can improve o1-minis biomedical reasoning through reinforcement learning (RL) finetuning. We show that RL finetuning of o1-mini results in large improvements in performance on gene classification, where it surprisingly outperformed domain-specific state-of-the-art models on some tasks. The results are mixed for small molecule prediction, suggesting that chemical reasoning could be more challenging for LLMs. We conclude with a discussion of the challenges and takeaways from this initial exploration of RL finetuning reasoning models for biomedical tasks.

bioinformatics↗

bioRxiv2025

In vivo CRISPR screening identifies NF1/RASA1/TP53 co-mutations and downstream MEK signaling as a common key mechanism of sinonasal tumorigenesis

Genomic alterations driving tumorigenesis in sinonasal malignancies remain largely unexplored. Here, we perform an in vivo loss-of-function screen using a pooled custom single-guide library delivered to the sinonasal cavity by adeno-associated virus vector to identify cancer driver genes across diverse sinonasal malignancies. This approach yielded sinonasal malignancies with diverse histologies, including sinonasal squamous cell carcinoma, adenocarcinoma, poorly differentiated sinonasal carcinoma, and sinonasal neuroendocrine tumors characteristic of olfactory neuroblastoma. Surprisingly, rather than observing distinct sgRNA profiles across sinonasal tumor subtypes, common recurrent mutations were identified in Nf1 (79%), Rasa1 (74%), and Trp53 (68%) across malignancies with distinct histologies. Utilizing an orthogonal approach, we confirmed that Nf1/Trp53 were required for sinonasal tumorigenesis. Given that loss-of-function in NF1 and RASA1 may lead to increased Ras activity and downstream MEK signaling, we tested small molecule targeting of the RAS-MAPK pathway in sinonasal malignancies. Indeed, both tumor cell lines derived from our loss-of-function approach as well as from human sinonasal malignancies displayed significant sensitivity to MEK inhibition in standard in vitro culture and organoid models. These findings demonstrate that loss of NF1 and RASA1-mediated Ras-GAP activity leads to Ras activation and downstream MEK signaling which is a potential common target throughout major sinonasal tumor subtypes.

cancer biology↗

bioRxiv2025

SyntheMol-RL: a flexible reinforcement learning framework for designing novel and synthesizable antibiotics

The rise of antibiotic-resistant pathogens such as Staphylococcus aureus has created an urgent need for new antibiotics. Generative artificial intelligence (AI) has shown promise in drug discovery, but existing models often fail to propose compounds that are both effective and synthetically tractable. To address these challenges, we introduce SyntheMol-RL, a reinforcement learning-based generative model that can rapidly design synthetically accessible small molecule drug candidates from a massive chemical space of 46 billion compounds. SyntheMol-RL improves upon our prior Monte Carlo Tree Search (MCTS)-based SyntheMol model by generalizing across chemically similar building blocks and enabling multi-parameter optimization. We applied SyntheMol-RL to generate candidate antibiotics against S. aureus by optimizing for both antibacterial activity and aqueous solubility, and we found that SyntheMol-RL generated molecules with improved predicted properties compared to both the previous MCTS version of SyntheMol as well as an AI-based virtual screening baseline. We synthesized 79 SyntheMol-RL compounds that were unique relative to the training dataset and found that 13 showed potent in vitro activity, of which seven were structurally novel after detailed literature searches. Furthermore, one hit compound, synthecin, demonstrated efficacy in a murine wound infection model of methicillin-resistant S. aureus (MRSA). These results validate SyntheMol-RLs ability to generate novel and synthetically accessible candidate antibiotics and position SyntheMol-RL as a powerful tool for drug design across therapeutic domains.

biochemistry↗

bioRxiv2025

A foundation model for protein-ligand affinity prediction through jointly optimizing virtual screening and hit-to-lead optimization

The structure of the protein binding pocket governs the ligand binding affinity by providing crucial intermolecular interactions and spatial compatibility. While existing methods have leveraged these structural insights to advance affinity prediction, they often treat virtual screening and hit-to-lead optimization separately, mainly due to incompatible speed-accuracy requirements. However, these two tasks complement each other, and their integration enables broader chemical exploration while preserving focus on affinity-determining substructures. Here, we present LigUnity, a foundation model for affinity prediction that jointly embeds ligands and pockets into a shared space. In particular, LigUnity learns coarse-grained active/inactive distinction through scaffold discrimination and fine-grained pocket-specific ligand preference through pharmacophore ranking. We demonstrate the effectiveness and versatility of LigUnity on eight benchmarks across six settings. In virtual screening, LigUnity outperforms 24 methods with >50% improvement and demonstrates robust generalization to novel targets. In hit-to-lead optimization, it achieves state-of-the-art performance across split-by-time, split-by-scaffold, and split-by-unit settings, emerging as a cost-efficient alternative to free energy perturbation. We further showcase how LigUnity can be employed in an active learning framework for TYK2 to efficiently find optimal ligands. Collectively, these results establish LigUnity as a versatile foundation model for affinity prediction, offering broad applicability across the drug discovery pipeline.

bioinformatics↗

bioRxiv2025

CHIP protects lysosomes from CLN4 mutant-induced membrane damages

Understanding how cells mitigate lysosomal damage is critical for unraveling pathogenic mechanisms of lysosome-related diseases. Here we use organelle-specific proteomics in iPSC-derived neurons (i3Neuron) and an in vitro lysosome-damaging assay to demonstrate that lysosome damage, caused by the aggregation of Ceroid Lipofuscinosis Neuronal 4 (CLN4)-linked DNAJC5 mutants on lysosomal membranes, serves as a critical pathogenic linchpin in CLN4-associated neurodegeneration. Intriguingly, in non-neuronal cells, a ubiquitin-dependent microautophagy mechanism downregulates CLN4 aggregates to counteract CLN4-associated lysotoxicity. Genome-wide CRISPR screens identify the ubiquitin ligase CHIP as a central microautophagy regulator that confers ubiquitin-dependent lysosome protection. Importantly, CHIPs lysosome protection function is transferrable, as ectopic CHIP improves lysosomal function in CLN4 i3Neurons, and effectively alleviates lipofuscin accumulation and neurodegeneration in a Drosophila CLN4 disease model. Our study establishes CHIP-mediated microautophagy as a key organelle damage guardian that preserves lysosome integrity, offering new insights into therapeutic development for CLN4 and other lysosome-related neurodegenerative diseases.

cell biology↗

bioRxiv2025

RAPID: Reliable and efficient Automatic generation of submission rePortIng checklists with large language moDels

ImportanceMedical reporting guidelines are significant in improving the transparency, quality, and integrity of medical research, particularly in randomized clinical trials; adherence to these guidelines supports research interpretability and has direct implications for downstream applications, such as patient treatment. However, with over 600 distinct reporting guidelines, manual assessments are often time-consuming and labor-intensive. ObjectiveTo evaluate an automated reporting checklist generation tool using large language models and retrieval augmentation generation technology, called RAPID. Design, Setting, and ParticipantsThis study used large language models to design a retrieval augmentation generation architecture and collected published journal articles as training and validation sets to optimize prompts within the framework and comprehensively evaluate the performance of the framework. Medical reporting experiments were collected from 50 randomized controlled trials without the intervention of AI tools and 41 randomized controlled trials with the intervention of AI tools. Main Outcomes and MeasuresFor effective evaluation of the performance of this tool, a classification accuracy metric (Reported/Not Reported) defined as the number of correct judgments divided by all judgments and a content consistent score metric defined as the number of contents retrieved by the tool that are the same as those retrieved by researchers divided by the total number of judgments were calculated. ResultsThe RAPID tool uses the widely used Word document and Portable Document Format as an input file. Fifty published journal articles without the intervention of AI tools and 41 published journal articles with the intervention of AI tools were collected as CONSORT and CONSORT-AI datasets. All of the CONSORT reporting items (37) were included in the tool. RAPID achieved a high average accuracy rate of 92.11% and a content consistency score of 81.14% on the CONSORT dataset. Of the CONSORT-AI reporting items, 11 items related to the intervention of AI tools were included in the tool. RAPID achieved an average accuracy of 83.81% with a content consistency score of 72.51% on the CONSORT-AI dataset. For these two reporting guidelines, a training set of 5 articles was selected from each dataset to refine the prompts used in the tools for CONSORT and CONSORT-AI reporting checklist. The validation set of the remaining articles was used to assess the performance of the RAPID. The RAPID tool used the Word document and Portable Document Format of the articles as input files. A RAPID graphical user interface was built using JavaScript and Vue. Conclusions and RelevanceThe RAPID tool is designed to assist in the reporting of various types of trials. RAPID has strong scalability, which can be easily adapted to different medical reporting guidelines without transfer learning on a large dataset. RAPID may effectively save time and improve working efficiency for different user groups, for example, 1) simplifying the submission process and improving report quality by verifying manuscript completeness for medical authors; 2) facilitating evaluation of report quality for medical researchers; 3) expediting manuscript distribution for medical editors; and 4) identifying reporting deficiencies and providing deeper insights for review comments for reviewers. Key PointsO_ST_ABSQuestionC_ST_ABSCan large language model tools automatically generate medical reporting checklists for manuscripts of different types of clinic trials? FindingsAn automated reporting checklist generation tool using large language models and retrieval augmentation generation technology, RAPID, was developed. RAPID was fully evaluated on all items across two separate datasets related to the Consolidated Standards of Reporting Trials (CONSORT) and CONSORT-AI reporting guidelines. In the first dataset corresponding to CONSORT, RAPID achieved an average accuracy of 92.11%, while in the second dataset associated with CONSORT - AI, it reached an average accuracy of 83.81%. Additionally, RAPID is highly scalable. It can be easily and smoothly adapted to different medical reporting guidelines without transfer learning on a large dataset. MeaningRAPID can effectively save time and improve working efficiency for different user groups such as medical authors, researchers, editors, and reviewers.

bioinformatics↗