Evaluating AI-Assisted Customer Verification for Synthetic Nucleic Acid Screening

Abstract

Legitimacy screening, the process of verifying the identity and purpose of customers ordering synthetic nucleic acids, is a primary safeguard against the misuse of synthetic biology. However, the associated costs discourage the adoption of screening practices. To evaluate whether AI tools can facilitate this process, we tested five large language models on five verification tasks using customer profiles of life sciences researchers from around the world. We compared AI performance against an expert human baseline on flag accuracy, source quality, source fidelity, and cost. Flag accuracy of the best-performing model (Gemini 2.5 Pro with four bibliographic and sanctions APIs) was statistically indistinguishable from the human baseline at 90.2% and 89.0% (n = 41). Gemini 2.5 Pro performed at or above the human baseline on source quality and fidelity, at roughly one-tenth of the cost ($1.18 vs. $14.04 per customer). For information-gathering tasks, which excluded the human review step, costs averaged $0.23 per customer, around 50 times cheaper than human screening. These results support piloting AI assistance at the information-gathering step of legitimacy screening at providers of synthetic nucleic acids and other dual-use biotechnology products, with human reviewers retaining authority over follow-up communication and order fulfillment decisions.