Search bioRxiv⌕ Search

Biology subjects

Vernon, R. M.

Publications and source records attributed to Vernon, R. M..

5 recordsLinked to original sources

pLM representations unlock metagenomic space beyond homology

Metagenomic sequencing has uncovered billions of proteins from uncultured microorganisms, vastly expanding the known protein space. Yet most remain functionally inaccessible because existing annotation methods depend on close homologs or accurate structure predictions. Here, we show that protein language models (pLMs) can unlock this diversity only when their training data are appropriately curated. We introduce Residue Embedding Diversity (RED), a metric for protein quality assessment orders of magnitude cheaper than likelihood, and a calibration task that measures model alignment with natural evolutionary distributions. We discover a fundamental trade-off between evolutionary calibration and structural modeling, establishing training data composition as a primary determinant of pLM behavior. Finally, we successfully retrieve diverse enzyme candidates from billions of metagenomic sequences and validate their expression in vivo.

bioinformatics↗

High-resolution dissection of concept acquisition in different families of protein language models

Protein language models have been increasingly successful on tasks ranging from fitness prediction to functional design, yet what biological knowledge they acquire and where it is encoded within their internal representations remain underexplored. Through a high-resolution layer-by-layer interpretability analysis of 8 models from the ESM2 and AMPLIFY families on 22 concepts from human proteome annotations, we found that these models encode concepts of increasing levels of complexity along their depth: basic physicochemical properties and linear motifs are best captured by early-layer embeddings, secondary structure from subsequent layers, and domain-level semantics from middle layers. Principal component projections of these embeddings showed that they separate biologically meaningful protein groupings, and molecular-biology-inspired interventions demonstrated that pLM embeddings can discriminate phosphomimic-active from inactive mutants. Perhaps surprisingly, we observed that pretraining data and compute had a greater impact on the linear emergence of biological concepts than scaling up parameters. By revealing where biological knowledge is captured in pLMs and which choices shape its emergence, our work offers insights to develop more robust, biologically grounded protein language models.

bioinformatics↗

Protein Language Models: Is Scaling Necessary?

Public protein sequence databases contain samples from the fitness landscape explored by nature. Protein language models (pLMs) pre-trained on these sequences aim to capture this landscape for tasks like property prediction and protein design. Following the same trend as in natural language processing, pLMs have continuously been scaled up. However, the premise that scale leads to better performance assumes that source databases provide an accurate representation of the underlying fitness landscape, which is likely false. By developing an efficient codebase, designing a modern architecture, and addressing data quality concerns such as sample bias, we introduce AMPLIFY, a best-in-class pLM that is orders of magnitude less expensive to train and deploy than previous models. Furthermore, to support the scientific community and democratize the training of pLMs, we have open-sourced AMPLIFYs pre-training codebase, data, and model checkpoints.

molecular biology↗

An interpretable machine learning algorithm to predict disordered protein phase separation based on biophysicalinteractions

Protein phase separation is increasingly understood to be an important mechanism of biological organization and biomaterial formation. Intrinsically disordered protein regions (IDRs) are often significant drivers of protein phase separation. A number of protein phase separation prediction algorithms are available, with many specific for particular classes of proteins and others providing results that are not amenable to interpretation of contributing biophysical interactions. Here we describe LLPhyScore, a new predictor of IDR-driven phase separation, based on a broad set of physical interactions or features. LLPhyScore uses sequence-based statistics from the RCSB PDB database of folded structures for these interactions, and is trained on a manually curated set of phase separation driver proteins with different negative training sets including the PDB and human proteome. Competitive training for a variety of physical chemical interactions shows the greatest importance of solvent contacts, disorder, hydrogen bonds, pi-pi contacts, and kinked-beta structure, with electrostatics, cation-pi, and absence of helical secondary structure also contributing. LLPhyScore has strong phase separation prediction recall statistics and enables a quantitative breakdown of the contribution from each physical feature to a sequences phase separation propensity. The tool should be a valuable resource for guiding experiment and providing hypotheses for protein function in normal and pathological states, as well as for understanding how specificity emerges in defining individual biomolecular condensates. Graphical Abstract O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=101 SRC="FIGDIR/small/499043v1_ufig1.gif" ALT="Figure 1"> View larger version (18K): org.highwire.dtl.DTLVardef@151f0cdorg.highwire.dtl.DTLVardef@984335org.highwire.dtl.DTLVardef@645b3dorg.highwire.dtl.DTLVardef@a22b06_HPS_FORMAT_FIGEXP M_FIG C_FIG

biophysics↗

IDPConformerGenerator: A Flexible Software Suite for Sampling Conformational Space of Disordered Protein States

The power of structural information for informing biological mechanism is clear for stable folded macromolecules, but similar structure-function insight is more difficult to obtain for highly dynamic systems such as intrinsically disordered proteins (IDPs) which must be described as structural ensembles. Here we present IDPConformerGenerator, a flexible, modular open source software platform for generating large and diverse ensembles of disordered protein states that builds conformers that obey geometric, steric and other physical restraints on the input sequence. IDPConformerGenerator samples backbone phi ({varphi}), psi ({psi}), and omega ({omega}) torsion angles of relevant sequence fragments from loops and secondary structure elements extracted from folded protein structures in the RCSB Protein Data Bank, and builds side chains from robust Monte Carlo algorithms using expanded rotamer libraries. IDPConformerGenerator has many user-defined options enabling variable fractional sampling of secondary structures, supports Bayesian models for assessing agreement of IDP ensembles for consistency with experimental data, and introduces a machine learning approach to transform between internal to Cartesian coordinates with reduced error. IDPConformerGenerator will facilitate the characterization of disordered proteins to ultimately provide structural insights into these states that have key biological functions.

biophysics↗