Biology subjects

Heinzinger, M.

Publications and source records attributed to Heinzinger, M..

At least 19 recordsLinked to original sources

bioRxiv2026

Translating chemical reactions into enzyme sequences enables formate dehydrogenase engineering

Enzyme engineering has broad applications across chemistry and biotechnology but remains costly and susceptible to low success rates. Generative artificial intelligence can substantially expand the sequence space accessible to enzyme engineering; however, directing generation towards enzymes that catalyse a user-defined chemical reaction remains challenging. Inspired by machine translation, we formulate enzyme generation as translation between two molecular languages: chemical reactions and amino acid sequences. Here, we introduce REXzyme, a translation model that generates enzymes conditioned on user-defined reactions. First, we evaluate its designs using multiple in silico metrics and characterize the strengths and limitations of its generalisation capacity. Then, we steer REXzyme towards the generation of enhanced formate dehydrogenases for their reverse natural reaction, i.e., the reduction of carbon dioxide. All 20 generated enzymes were expressed and purified as soluble proteins and exhibited CO2-reduction activity, with the most active design reaching 2.5-fold the specific activity of the reference enzyme. Notably, all substitutions in the designed variants were performed on distal positions outside the catalytic site. Our results show that reaction-to-sequence translation provides a route for efficiently engineering enhanced biocatalysts.

biochemistry↗

bioRxiv2026

Protein Language Models: Learning From Evolution, Designing Beyond It

Protein language models (PLMs) have transformed our ability to learn from evolutionary sequence space, but protein engineering ultimately asks a different question: not what evolution selected, but what we should build next. Zero-shot likelihoods therefore provide useful, but not universal, measures of fitness and can misalign with engineering objectives. Experimental supervision redirects these priors towards properties of interest, enabling target-specific prediction and closed-loop optimization. PLMs thereby complement structure-based design: structural methods provide geometric control, while PLMs integrate experimental feedback to optimize functional and developability properties. Realizing this potential requires evaluation beyond retrospective predictive accuracy, towards extrapolation, multi-objective optimization, and prospective experimental success. We argue for recurring competitions combining scalable predictive benchmarks with prospective experimental challenges. Ultimately, progress should be measured not by predicting existing experiments, but by enabling successful new ones.

bioinformatics↗

bioRxiv2026

ContrasTED: contrastive domain embeddings for scalable remote homology classification

Protein structure prediction has expanded structural databases to hundreds of millions of domains. Classifying these domains into homologous superfamilies reveals evolutionary and functional relationships that can persist despite low sequence similarity. As the size of structural databases continues to grow, homology classification requires methods that combine scalability with accuracy. Here we present ContrasTED, which uses CATH-supervised center-contrastive learning to project structure-aware embeddings into a domain-level metric space for nearest-centroid superfamily assignment. On a sequence-filtered S20 benchmark (n = 1,028), superfamily assignment accuracy reached 92.9% (1-NN) and 91.4% (nearest centroid), exceeding sequence search, profile HMMs, Foldseek, and a classifier trained on embeddings. The learned latent space separates superfamilies while retaining structural information below 20% sequence identity, with the largest gains among sparsely represented superfamilies. ContrasTED produces 4.67 million new candidate assignments across 3,796 superfamilies in The Encyclopedia of Domains (TED), extending annotation coverage beyond previous structure-based methods.

bioinformatics↗

bioRxiv2026

DeePEn - A Depth sensitive benchmark for Protein Engineering

Recent progress in modeling techniques and high-throughput screening has significantly enhanced the accessibility of protein engineering. Nevertheless, further progress gets hindered by the lack of robust benchmarks that capture the practical challenges for real-world protein engineering. Here, we introduced DeePEn, a Depth-sensitive benchmark for Protein Engineering that quantifies a models generalization capabilities when predicting protein fitness at increasing mutational distance from the wildtype or training data. We defined distance as the number of simultaneous point mutations, i.e., single amino acid variants (SAVs), moving from wild-type to mutant (edit distance in computer science jargon). Specifically selecting four deep mutational scanning (DMS) datasets with sufficient multi-mutation data points from ProteinGym, we assessed recent predictive models, including general and biophysics-informed protein Language Models (pLMs), and a non-transformer neural network. Our results highlight how the performance of all models deteriorates with increasing mutational distance and that no single metric sufficiently captures the diverse requirements of protein engineering. To overcome these shortcomings, DeePEn provides a readily available resource for multi-metric benchmarking that focuses on the prediction of distant variants.

bioinformatics↗

bioRxiv2026

Pseudoperplexity Probes Memorization in Protein Language Models

Protein Language Models (pLMs) have significantly advanced computational biology. Yet their scale and reliance on redundant training data raise a fundamental question: do pLMs generalize the statistical grammar of proteins, or do they simply memorize their training data? To investigate this, we used pseudoperplexity as a probe for sequence-level memorization, comparing ProtT5's pseudoperplexity on a pre-training proxy dataset against a post-training holdout of genuinely novel sequences. To ensure a valid comparison, we matched the datasets by sequence length, cluster size, and taxonomic family. As a statistical baseline, we trained n-gram language models; analysis of higher-order n-gram composition and a statistically significant divergence in perplexity confirmed that the post-training sequences were genuinely novel at the local sequence level. ProtT5 showed a statistically significant difference in pseudoperplexity between seen and unseen sequences, though further analysis revealed this memorization signal to be modest. These findings suggest that ProtT5 exhibits detectable but limited memorization of its training data as measured by a pseudoperplexity-based probe.

bioinformatics↗

bioRxiv2026

Adaptive and Spandrel-like Constraints at Functional Sites in Protein Folds

How new molecular functions emerge during protein evolution remains a fundamental question in molecular biology. The energy landscape theory states that proteins are minimally frustrated, i.e. they have minimized their internal conflicts, to allow robust folding. Yet, not all energetic conflicts are eliminated, with functional regions such as catalytic residues and ligand-binding sites being often enriched in frustrated interactions, trading localized stability for biological activity. However, it is still uncertain whether this functional frustration is an evolutionary adaptation, positively selected despite its energetic cost or an inevitable physical byproduct of the fold architecture. Here, we combine reverse folding, structure prediction, and sequence analysis with local frustration profiling to address this long-standing question. Unexpectedly, we found that reverse folding algorithms are unable to energetically minimize evolutionary conserved frustration at specific residues, even when detrimental to overall structural stability. We propose that these frustration hotspots act as architectural spandrels, inherent physical constraints of the fold that evolution subsequently co-opts for function. Our findings connect biophysical constraints and evolutionary selection, providing a new framework to understand how functional specificity emerges in protein landscapes.

bioinformatics↗

bioRxiv2026

FrustrAI-Seq: Scaling Local Energetic Frustration to the Protein Sequence Space

Proteins fold into their native three-dimensional (3D) structures by navigating complex energy landscapes shaped by the biophysical and biochemical properties of their sequence. Once folded, some sequence positions (dubbed residues) remain locally frustrated, reflecting functional constraints incompatible with optimal packing. This local energetic frustration provides important insights into protein function and dynamics, but its analysis typically relies on structure-based energy calculations and remains energetically costly at scale. Here, we introduce an ultra-fast sequence-based prediction of local energetic frustration directly from protein sequences using embeddings from protein language models (pLMs). Our method, coined FrustrAI-Seq, enables proteome-wide frustration profiling in minutes (17 minutes for the entire human proteome on a single Nvidia H100 GPU) while retaining biologically relevant performance as shown for the alpha-globin and beta-lactamase family. By eliminating the need for explicit structural or evolutionary information, this approach expands frustration analysis to protein regions and classes that were previously inaccessible, including intrinsically disordered regions and high-throughput de novo designed protein datasets. To support reproducibility and large-scale applications, we provide the largest freely available resource of precomputed local frustration scores to date (10^6 proteins), along with model weights and complete training and inference code at: github.com/leuschjanphilipp/FrustrAI-Seq.

bioinformatics↗

bioRxiv2026

Protein Language Modeling beyond static folds reveals sequence-encoded flexibility

MotivationProteins function through motion. Yet, most discoveries still commence with static representations of protein structures. Here, we investigated the feasibility of leveraging protein dynamics to improve homology detection. ResultsWe introduce ProtProfileMD, a sequence-to-3D-probability model that predicts, from an amino acid sequence, a profile of discrete structural representations capturing protein dynamics. We applied supervised parameter-efficient finetuning of the ProstT5 protein Language Model (pLM) to predict per-residue distributions over Foldseeks 3Di alphabet derived from motions observed in molecular dynamics. This original result reveals that the 3Di tokens, despite being coarse-grained descriptors of 3D structure, still offer sufficient resolution to capture aspects of conformational changes. This is evidenced by a correlation between fluctuations in the 3D protein structure over the course of a molecular dynamics trajectory and the entropy of 3Di states. Based on this insight, we introduce a proof-of-concept for making remote homology detection of proteins more sensitive by leveraging a proteins distinctive dynamic fingerprint captured by our model. Our method recovers flexibility signals with a fidelity that is biologically relevant, improving search and complementing protein structure predictions, for example, by flagging flexible, disordered, or other functionally relevant regions. Availability and ImplementationProtProfileMD is available at github.com/finnlueth/ProtProfileMD. The associated training data and model weights are available at huggingface.co/datasets/finnlueth/ProtProfileMD and huggingface.co/finnlueth/ProtProfileMD.

bioinformatics↗

bioRxiv2025

ProFam: Open-Source Protein Family Language Modelling for Fitness Prediction and Design

Protein language models have become essential tools for engineering novel functional proteins. The emerging paradigm of family-based language models makes use of homologous sequences to steer protein design and enhance zero-shot fitness prediction, by imbuing models with an ability to explicitly reason over evolutionary context. To provide an open foundation for this modelling approach, we introduce ProFam-1, a 251M-parameter autoregressive protein family language model (pfLM) trained with next-token prediction on millions of protein families represented as concatenated, unaligned sets of sequences. ProFam-1 is competitive with state-of-the-art models on the ProteinGym zero-shot fitness prediction benchmark, achieving Spearman correlations of 0.47 for substitutions and 0.53 for indels. For homology-guided generation, ProFam-1 generates diverse sequences with predicted structural similarity, while preserving residue conservation and covariance patterns. All of ProFams training and inference pipelines, together with our curated, large-scale training dataset ProFam Atlas, are released fully open source, lowering the barrier to future method development.

bioinformatics↗

bioRxiv2025

Protein Structure Informed Bacteriophage Genome Annotation with Phold

Bacteriophage (phage) genome annotation is essential for understanding their functional potential and suitability for use as therapeutic agents. Here we introduce Phold, an annotation framework utilising protein structural information that combines the ProstT5 protein language model and structural alignment tool Foldseek. Phold assigns annotations using a database of over 1.36 million predicted phage protein structures with high quality functional labels. Benchmarking reveals that Phold outperforms existing sequence-based homology approaches in functional annotation sensitivity whilst maintaining speed, consistency and scalability. Applying Phold to diverse cultured and metagenomic phage genomes shows it consistently annotates over 50% of genes on an average phage and 40% on an average archaeal virus. Comparisons of phage protein structures to other protein structures across the tree of life reveals that phage proteins commonly have structural homology to proteins shared across the tree of life, particularly those that have nucleic acid metabolism and enzymatic functions. Phold is available as free and open-source software at https://github.com/gbouras13/phold.

bioinformatics↗

bioRxiv2024

ProtSpace: a tool for visualizing protein space

Protein language models (pLMs) generate high-dimensional representations of proteins, so called embeddings, that capture complex information stored in the set of evolved sequences. Interpreting these embeddings remains an important challenge. ProtSpace provides one solution through an open-source Python package that visualizes protein embeddings interactively in 2D and 3D. The combination of embedding space with protein 3D structure view aids in discovering functional patterns readily missed by traditional sequence analysis. We present two examples to showcase ProtSpace. First, investigations of phage data sets showed distinct clusters of major functional groups and a mixed region, possibly suggesting bias in todays protein sequences used to train pLMs. Second, the analysis of venom proteins revealed unexpected convergent evolution between scorpion and snake toxins; this challenges existing toxin family classifications and added evidence refuting the aculeatoxin family hypothesis. ProtSpace is freely available as a pip-installable Python package (source code & documentation) with examples on GitHub (https://github.com/tsenoner/protspace) and as a web interface (https://protspace.rostlab.org). The platform enables seamless collaboration through portable JSON session files.

bioinformatics↗

bioRxiv2024

TMVisDB: A Resource for Transmembrane Protein Annotations and 3D Visualizations

Since the rise of cellular life, transmembrane proteins (TMPs) have been crucial to various cellular processes through their central role as gates and gatekeepers. Despite their importance, experimental high-resolution structures for TMPs remain underrepresented due to experimental challenges. Given its performance leap, structure predictions have begun to close the gap. However, identifying the membrane regions and topology in three-dimensional structure files on a large scale still requires additional in silico predictions. Here, we introduce TMVisDB to sieve through millions of predicted structures for TMPs. This resource enables both browsing through 46 million predicted TMPs and visualizing them along with their topological annotations without having to tap into costly predictions of the AlphaFold3-style. TMVisDB joins AlphaFoldDB structure predictions and transmembrane topology predictions from the protein language model (pLM) based method TMbed. We showcase the utility of TMVisDB for the analysis of proteins through two use cases, namely the B-lymphocyte antigen CD20 (Homo sapiens) and the cellulose synthase (Novosphingobium sp. P6W). We demonstrate the value of TMVisDB for large-scale analyses through findings pertaining to all TMPs predicted for the human proteome. TMVisDB is freely available at https://tmvisdb.rostlab.org

bioinformatics↗

bioRxiv2024

Protein Embeddings Predict Binding Residues in Disordered Regions

The identification of protein binding residues helps to understand their biological processes as protein function is often defined through ligand binding, such as to other proteins, small molecules, ions, or nucleotides. Methods predicting binding residues often err for intrinsically disordered proteins or regions (IDPs/IDPRs), often also referred to as molecular recognition features (MoRFs). Here, we presented a novel machine learning (ML) model trained to specifically predict binding regions in IDPRs. The proposed model, IDBindT5, leveraged embeddings from the protein language model (pLM) ProtT5 to reach a balanced accuracy of 57.2{+/-}3.6% (95% confidence interval). Assessed on the same data set, this did not differ at the 95% CI from the state-of-the-art (SOTA) methods ANCHOR2 and DeepDISOBind that rely on expert-crafted features and evolutionary information from multiple sequence alignments (MSAs). Assessed on differ data, methods such as SPOT-MoRF reached higher MCCs. IDBindT5s SOTA predictions are much faster than other methods, easily enabling full-proteome analyses. Our findings emphasize the potential of pLMs as a promising approach for exploring and predicting features of disordered proteins. The model and a comprehensive manual are publicly available at https://github.com/jahnl/binding_in_disorder.

bioinformatics↗

bioRxiv2023

Fine-tuning protein language models boosts predictions across diverse tasks

Prediction methods inputting embeddings from protein Language Models (pLMs) have reached or even surpassed state-of-the-art (SOTA) performance on many protein prediction tasks. In natural language processing (NLP) fine-tuning large Language Models (LLMs) has become the de facto standard. In contrast, most pLM-based protein predictions do not back-propagate to the pLM. Here, we compared the fine-tuning of three SOTA pLMs (ESM2, ProtT5, Ankh) on eight different tasks. Two results stood out. Firstly, task-specific supervised fine-tuning almost always improved downstream predictions. Secondly, parameter-efficient fine-tuning could reach similar improvements consuming substantially fewer resources at up to 4.5-fold acceleration of training over fine-tuning full models. Our results suggested to always try fine-tuning, in particular for problems with small datasets, such as for fitness landscape predictions of a single protein. For ease of adaptability, we provided easy-to-use notebooks to fine-tune all models used during this work for per-protein (pooling) and per-residue prediction tasks at https://github.com/RSchmirler/data-repo_plm-finetune-eval.

bioinformatics↗

bioRxiv2023

ProstT5: Bilingual Language Model for Protein Sequence and Structure

Adapting large language models (LLMs) to protein sequences spawned the development of powerful protein language models (pLMs). Concurrently, AlphaFold2 broke through in protein structure prediction. Now we can systematically and comprehensively explore the dual nature of proteins that act and exist as three-dimensional (3D) machines and evolve as linear strings of one-dimensional (1D) sequences. Here, we leverage pLMs to simultaneously model both modalities by combining 1D sequences with 3D structure in a single model. We encode protein structures as token sequences using the 3Di-alphabet introduced by the 3D-alignment method Foldseek. This new foundation pLM extracts the features and patterns of the resulting "structure-sequence" representation. Toward this end, we built a non-redundant dataset from AlphaFoldDB and fine-tuned an existing pLM (ProtT5) to translate between 3Di and amino acid sequences. As a proof-of-concept for our novel approach, dubbed Protein structure-sequence T5 (ProstT5), we showed improved performance for subsequent prediction tasks, and for "inverse folding", namely the generation of novel protein sequences adopting a given structural scaffold ("fold"). Our work showcased the potential of pLMs to tap into the information-rich protein structure revolution fueled by AlphaFold2. ProstT5 paves the way to develop new tools integrating the vast resource of 3D predictions, and opens new research avenues in the post-AlphaFold2 era. Our model is freely available for all at https://github.com/mheinzinger/ProstT5.

bioinformatics↗

bioRxiv2022

Domain loss enables evolution of novel functions in a gene superfamily, including snake 3-finger toxins

Three-finger toxins (3FTxs) are a functionally diverse family of toxins, apparently unique to venoms of caenophidian snakes. Although the ancestral function of 3FTxs is antagonism of nicotinic acetylcholine receptors (nAChR), redundancy conferred by accumulation of duplicate genes has facilitated extensive neofunctionalization, such that derived members of the family interact with a range of targets. 3FTxs are members of the Ly6/uPAR family, but their non-toxin ancestor remains unknown. Combining traditional phylogenetic approaches, manual synteny analysis, and cutting-edge machine learning techniques (including AlphaFold2 and ProtT5), we have reconstructed a detailed evolutionary history of 3FTxs. We identify their immediate ancestor as a non-secretory Ly6, unique to squamate reptiles, and propose that changes in molecular ecology resulting from loss of a membrane-anchoring domain (MaD) and changes in gene expression, paved the way for evolution of one of the most important families of snake toxins.

molecular biology↗

bioRxiv2022

TMvisDB: resource for transmembrane protein annotation and 3D visualization

Since the rise of cellular organisms, transmembrane proteins (TMPs) have been crucial to a variety of cellular processes due to their central role as gates and gatekeepers. Despite their importance, experimental high-resolution structures for TMPs remain underrepresented due to technical limitations. With structure prediction methods coming of age, predictions might fill some of the need. However, identifying the membrane regions and topology in three-dimensional structure files requires additional in silico prediction. Here, we introduce TMvisDB to sieve through millions of predicted structures for TMPs. This resource enables both, to browse through 46 million predicted TMPs and to visualize those along with their topological annotations. The database was created by joining AlphaFold DB structure predictions and transmembrane topology predictions from the protein language model based method TMbed. We show the utility of TMvisDB for individual proteins through two single use cases, namely the B-lymphocyte antigen CD20 (Homo sapiens) and the cellulose synthase (Novosphingobium sp. P6W). To demonstrate the value for large scale analyses, we focus on all TMPs predicted for the human proteome. TMvisDB is freely available at tmvis.predictprotein.org.

bioinformatics↗

bioRxiv2022

Ultra-fast protein structure prediction to capture effects of sequence variation in mutation movies

Top protein three-dimensional (3D) structure predictions require evolutionary information from multiple-sequence alignments (MSAs) and deep, convolutional neural networks and appear insensitive to small sequence changes. Here, we describe EMBER3D using embeddings from the pre-trained protein language model (pLM) ProtT5 to predict 3D structure directly from single sequences. Orders of magnitude faster than others, EMBER3D predicts average-length structures in milliseconds on consumer-grade machines. Although not nearly as accurate as AlphaFold2, the speed of EMBER3D allows a glimpse at future applications such as the almost real-time rendering of deep mutational scanning (DMS) movies that visualize the effect of all point mutants on predicted structures. This also enables live-editing of sequence/structure pairs. EMBER3D is accurate enough for highly sensitive rapid remote homology detection by Foldseek identifying structural similarities. Overall, our use cases suggest that speed can complement accuracy, in particular when accessible through consumer-grade machines. EMBER3D is free and publicly available: https://github.com/kWeissenow/EMBER3D.

bioinformatics↗