A germline shortcut in protein language model retrieval of adaptive immune receptors

Abstract

Protein language models (PLMs) are increasingly used to represent adaptive immune receptors; however, their advantages over classical alignment-based methods remain unclear. We benchmarked the ESM2 family, ESM-C, antibody-, and TCR-specific PLMs against sequence-alignment methods across four B-cell and T-cell receptor databases using CDR3, clonotype, paratope, and full variable domain representations. At the CDR3 level, alignments consistently out-performed all PLMs, except SCEPTR, a contrastively pre-trained TCR model that additionally encodes the germline V gene. BLOSUM62 and Levenshtein distance metrics exceeded all ESM2 variants by 7-12 percentage points in top-1 retrieval accuracy, and domain-specific PLMs did not close this gap. At the full variable domain level, the alignments and PLMs converged. Alignment performance peaked at the paratope level, whereas PLMs benefited primarily from the addition of conserved framework information in the full-length sequences. The one exception was heavily-mutated HIV-1 antibodies, where PLMs outperformed alignments. These results, together with germline reversion analyses, indicate that full-length PLM performance takes advantage of a germline shortcut rather than improved extraction of antigen-specific information from CDRs. We further showed that conventional random train-test splits inflate retrieval accuracy by 15-28 percentage points owing to clonal leakage. Together, these findings define the strengths and limitations of frozen zero-shot PLM embeddings for immune receptor retrieval and establish clone-aware benchmarking as a practical standard of comparison. Key PointsO_LIClassical sequence alignment consistently outperformed frozen protein language models (PLMs) for CDR3 immune-receptor retrieval across the four datasets. C_LIO_LIProtein language models reached parity only in the full variable domain by exploiting the conserved germline-derived sequence, thus revealing a germline shortcut. C_LIO_LIFrozen PLMs exceeded alignment only for highly mutated HIV-1 antibodies. C_LIO_LIOne PLM matching alignment at CDR3, SCEPTR, encoded germline V gene information. C_LIO_LIRandom train-test splitting inflated accuracy by 15-28 percentage points; clone-aware splitting should be the standard. C_LI Bibliographical NoteDaron M. Standley is Professor of Genome Informatics at the Research Institute for Microbial Diseases, Osaka University. His research developed computational methods for immune repertoire analysis and antibody discovery.