LLM-assisted analytical review catches statistical errors that human researchers miss: documented evidence from two computational biology case studies

Abstract

Large language models (LLMs) are increasingly used to assist with code generation and data analysis in computational biology, but their capacity to serve as analytical reviewers -- detecting statistical and methodological errors in completed analyses -- is less documented. We present two case studies in which LLM-assisted review identified consequential errors in computational biology analyses that the human researchers had not detected. In single-cell transcriptomics, review of a differential expression analysis of 22,048 cells from human striatum identified a sex composition confound (Chi-squared = 192.7, p = 8.1 x 10-44) that had produced spurious Y-chromosome gene hits; correction revealed KIF3A as the sole genuine finding (adjusted p = 1.37 x 10-14). In HRV-based stress classification, review identified subject leakage in a train-test split that allowed the same individuals in training and test sets; correction via leave-one-subject-out cross-validation produced a more reliable and higher AUC estimate (0.946 vs. 0.848). Both errors would have survived conventional peer review in the absence of systematic checking. Both corrected analyses have been submitted for peer review. These cases suggest that LLM-assisted analytical review -- using LLMs to interrogate completed analyses for confounds, leakage, and statistical artifacts -- is a practical quality assurance step that complements rather than replaces human scientific judgment.