Biology subjects

Yang, K. K.

Publications and source records attributed to Yang, K. K..

At least 19 recordsLinked to original sources

bioRxiv2026

Protein language models and the long tail of functional diversity

Protein language model performance on downstream tasks depends on the pretraining data, motivating recent efforts to combine genomic- and metagenomic-derived protein sequences into large-scale atlases. Because these datasets are highly redundant, sequences are typically clustered by similarity and sampled during training. Sequences that do not belong to any cluster, known as "singletons", are typically excluded from training and evaluation because they are considered to be artifacts. However, singletons represent the long tail of functional diversity and are abundant in many large-scale atlases: nearly 43% of the 3.34 billion sequences in the joint genomic-metagenomic dataset GigaRef are singletons. Here, we characterize singletons derived from UniRef and GigaRef by assessing whether clustering missed homologs, how much their exclusion affects protein language model (PLM) training, and which biological domains they contain. We find that many GigaRef singletons belong to a cluster under alternative parameter settings, suggesting that genomic and metagenomic datasets may require dataset-specific clustering configurations. We also show that singletons share mutual information with clustered sequences, making them learnable by PLMs and useful for training. Finally, metagenomic singletons carry denser, more diverse domain content than clustered sequences, including domain-level homology that sequence-identity clustering misses. Together, these results support including singletons in PLM training and call for closer examination of data curation in large-scale integrated sequence atlases.

bioinformatics↗

bioRxiv2026

Vermeer: Autoregressive generative modeling of microscopy predicts protein localization

Fluorescent microscopy provides a rich view into how proteins localize within cells, but it remains experimentally infeasible to image human proteins across all of the different factors that can impact localization. We introduce Vermeer, a channel-adaptive autoregressive generative model for in silico generation of microscopy images of protein localization. Vermeer conditions generations on protein sequences and landmark stains showing the morphology of cells, which enables it to generalize to unseen proteins and cell lines. We show that Vermeer, trained on the Human Protein Atlas, can generate images with substantially improved perceptual quality and biological fidelity over previous proposals. Additionally, Vermeers autoregressive framework enables flexible generation using varying channel subsets and orderings, enabling zero-shot transfer to data collected under different imaging conditions and channel configurations than those used for training. These results position Vermeer to enable scalable modeling of protein localization and is a step towards generative foundation models that can operate over distinct microscopy datasets. Code is publicly accessible at https://github.com/microsoft/vermeer.git.

cell biology↗

bioRxiv2026

Evolutionary conditioning enables guided generation of functionally diverse enhancers

Deep learning has been instrumental in our understanding of how enhancers encode regulatory information in their DNA sequence and has demonstrated preliminary success with enhancer design. However, the prevailing approach for enhancer design, cell type label conditioning, depends on labeled data from massively parallel reporter assays, which only exists for a handful of cell types. We propose EnhancAR, an autoregressive model trained on sets of unaligned homologous enhancer sequences to learn the function of the enhancer conserved over evolution and generate sequences that resemble real homologs. By training EnhancAR on 1.7 million human enhancer homolog sets spanning 1,888 cell types, EnhancAR generates enhancers for a variety of contexts without being conditioned on a cell type label. We computationally validate that when conditioned on a set of enhancer homologs, EnhancAR generates novel and diverse sequences that preserve the functional properties of the homologs. By prompting EnhancAR with homologs for existing cell type specific enhancers, we design enhancers with similar predicted cell type specificity. We further demonstrate that when trained on length sorted homologs, EnhancAR can design enhancers shorter than the conditioning homologs that preserve the predicted activity. In summary, we find that leveraging evolutionary information in enhancer homologs enables a more flexible and general paradigm for designing enhancers with specific functions.

bioengineering↗

bioRxiv2026

FLIP2: Expanding Protein Fitness Landscape Benchmarks for Real-World Machine Learning Applications

Machine learning methods that predict protein fitness from sequence remain sensitive to changes in data distributions, limiting generalization across common conditions encountered in protein engineering. Practically, protein engineers are thus left wondering about the effective utility of ML tools. The FLIP benchmark established protocols for testing generalization under some domain shifts, but it was limited to measurements of thermostability, binding, and viral capsid viability. We introduce FLIP2, a protein fitness benchmark spanning seven new datasets, including enzymes, protein-protein interactions, and light-sensitive proteins, as well as splits that measure generalization relevant to real-world protein engineering campaigns. Evaluating a suite of benchmark models across these datasets and splits reveals that simpler models often matched or outperformed fine-tuned protein language models on FLIP2, challenging the utility of existing transfer learning techniques. Provenance for all datasets has been recorded and we redistribute all data CC-BY 4.0 to facilitate continued progress.

bioengineering↗

bioRxiv2026

Predicting evolutionary rate as a pretraining task improves genome language model representations

Genome language models (gLM) have the potential to further understanding of regulatory genomics without requiring labeled data. Most gLMs are pretrained using sequence reconstruction tasks inspired by natural language processing, but recent studies have shown that these gLMs often fail to capture biological signal. To overcome this, we introduce pretraining tasks that predict the rate of evolution. These tasks are designed so that they can be composed with sequence reconstruction, enabling a controlled comparison of predicting sequence only, evolutionary rate only, or both. To address gaps in existing evaluations, we developed a suite of biologically grounded benchmarks. Across these tasks, and for established variant effect prediction benchmarks, models pretrained on both sequence and evolutionary rate outperform those trained on sequence alone, and training on evolutionary rate can make the even the relatively small models in our work competitive with much larger existing gLMs for some tasks. These results establish evolution as a key training target for genome-scale models.

genomics↗

bioRxiv2026

Designing AI-programmable therapeutics with the EDEN family of foundation models

The ability to interpret, modify, and design DNA has driven many of the most significant advances in modern medicine, from diagnostics, biologics, and vaccines to cell and gene therapies. However, the inherent complexity of biological systems means that most modern medicines are still engineered using bespoke, labor-intensive processes. To address the need for a generalisable and programmable approach to therapeutic design, we introduce the EDEN (environmentally-derived evolutionary network) family of metagenomic foundation models, including a 28 billion parameter model trained on 9.7 trillion nucleotide tokens from BaseData1. This dataset, at the time of training, contained more than 10 billion novel genes from over 1 million new species, and is intentionally enriched for environmental and host-associated metagenomes, phage sequences, and mobile genetic elements, enabling the model to learn from diverse and novel cross-species evolutionary mechanisms and apply them to key challenges in human health. EDEN achieves state-of-the-art performance across a series of predictive and generative genomic and protein benchmarks. To demonstrate the models broad applicability across biology, we evaluate EDENs capacity for programmable therapeutic design by challenging a single architecture to design biological novelty across three distinct therapeutic modalities, disease areas and biological scales: (i) large gene insertion, (ii) antibiotic peptide design, and (iii) microbiome design. First, we demonstrate AI-programmable Gene Insertion (aiPGI), in which EDEN designs de novo large serine recombinases (LSRs) capable of inserting large pieces of DNA at desired target sites in the human genome when prompted only on 30 nucleotides of DNA sequence from the desired target site. In low-N experimental validation, EDEN generated multiple active recombinases for all tested disease-associated genomic loci (ATM, DMD, F9, FANCC, GALC, IDS, P4HA1, PHEX, RYR2, USH2A) and 4 potential safe harbor sites in the human genome. EDEN achieves an overall functional hit rate of 63.2% across diverse DNA prompts when prompted on only 30bp of DNA from outside the training data. 50% of EDEN-generated LSRs were active in human cells, achieving therapeutically relevant levels of CAR insertion in primary human T cells. We also show that EDEN can generate active bridge recombinases when prompted on the associated guide RNA alone, with sequence identities to training and public data as low as 65%. These results pave the way for a new generation of cell and gene therapies by opening the door to rapid, programmable and site-specific integration of large genetic payloads without double-strand breaks. This offers an alternative to the safety, efficiency and payload limitations inherent in viral or nuclease-based editing at thousands of currently intractable human therapeutic targets. Second, we use the same model to generate a focused low-N library of novel antimicrobial peptides where 97% showed activity, with top candidates achieving single-digit micromolar potency against critical-priority multidrug-resistant pathogens. Third, to demonstrate that EDEN captures inter-genomic features, we design a gigabase-scale microbiome with over 94,000 synthetic metagenomic assemblies, including prophage genomes and correct cross-species metabolic pathway completions. The EDEN-generated synthetic microbiome covers 9,067 species with a biome-specific taxonomic accuracy of 99%. Over 1,500 of the generated species were outside the fine-tuning dataset while retaining the correct microecological properties and biome association, thus significantly expanding genetic and taxonomic diversity. Together, these results establish a new strategic direction for AI-programmable therapeutics, in which a single foundation model architecture designs candidate therapeutics across diverse modalities and disease areas. This suggests that the combination of billions of years of evolutionary data with specific therapeutic records offers a clear, scaling-driven path to making therapeutic design a predictable engineering discipline. O_FIG O_LINKSMALLFIG WIDTH=141 HEIGHT=200 SRC="FIGDIR/small/699009v2_ufig1.gif" ALT="Figure 1"> View larger version (59K): org.highwire.dtl.DTLVardef@68c20borg.highwire.dtl.DTLVardef@19b8e8corg.highwire.dtl.DTLVardef@1ab9362org.highwire.dtl.DTLVardef@1592cb7_HPS_FORMAT_FIGEXP M_FIG C_FIG

genomics↗

bioRxiv2025

The Dayhoff Atlas: scaling sequence diversity for improved protein generation

Organized information powers modern biology, a framework pioneered by Margaret Dayhoffs Atlas of Protein Sequence and Structure and advanced by todays databases and computational methods. Here, we extend this paradigm for the AI era, presenting the Dayhoff Atlas of protein sequence data and generative models to accelerate protein biology and design. The Atlas introduces GigaRef, the largest open dataset of natural proteins, spanning 3.34B genomic and metagenomic sequences across 1.70B clusters, and BackboneRef, which distills structural information from 240,811 synthetic backbones into 46M synthetic sequences. Leveraging these datasets, we trained the Dayhoff protein language models, which can predict mutation effects, scaffold structural motifs, and generate novel proteins within families. Training on metagenomic and structure-based synthetic sequences increased the expression rates of generated proteins, demonstrating the value of data diversity and scale. We release the Dayhoff Atlas code, datasets, and models under a permissive license to empower computation in protein design.

bioengineering↗

bioRxiv2025

Trainable subnetworks reveal insights into structure knowledge organization in protein language models

Protein language models (PLMs) pretrained via a masked language modeling objective have proven effective across a range of structure-related tasks, including high-resolution structure prediction. However, it remains unclear to what extent these models factorize protein structural categories among their learned parameters. In this work, we introduce trainable subnetworks, which mask out the PLM weights responsible for language modeling performance on a structural category of proteins. We systematically trained 39 PLM subnetworks targeting both sequence- and residue-level features at varying degrees of resolution using annotations defined by the CATH taxonomy and secondary structure elements. Using these PLM subnetworks, we assessed how structural factorization in PLMs influences downstream structure prediction. Our results show that PLMs are highly sensitive to sequence-level features and can predominantly disentangle extremely coarse or fine-grained information. Furthermore, we observe that structure prediction is highly responsive to factorized PLM representations and that small changes in language modeling performance can significantly impair PLM-based structure prediction capabilities. Our work presents a framework for studying feature entanglement within pretrained PLMs and can be leveraged to improve the alignment of learned PLM representations with known biological concepts.

bioinformatics↗

bioRxiv2025

Deep learning guided design of protease substrates

Proteases, a class of enzymes that play critical roles in health and disease, exert their function through the cleavage of peptide bonds. Identifying substrates that are efficiently and selectively cleaved by target proteases is essential for studying protease activity and for harnessing their activity in protease-activated diagnostics and therapeutics. However, the vast design space of possible substrates (c.a. 2010 unique amino acid combinations for a 10-mer peptide) and the limited accessibility of high-throughput activity profiling tools hinder the speed and success of substrate design. We present CleaveNet, an end-to-end AI pipeline for the design of protease substrates. Applied to matrix metalloproteinases, CleaveNet enhances the scale, tunability, and efficiency of substrate design. CleaveNet generates peptide substrates that exhibit sound biophysical properties and capture not only well-established but also novel cleavage motifs. To enable precise control over substrate design, CleaveNet incorporates a conditioning tag that enables generation of peptides guided by a target cleavage profile, enabling targeted design of efficient and selective substrates. CleaveNet-generated substrates were validated experimentally through a large-scale in vitro screen, even in the challenging case of designing highly selective substrates for MMP13. We envision that CleaveNet will accelerate our ability to study and capitalize on protease activity, paving the way for new in silico design tools across enzyme classes.

biochemistry↗

bioRxiv2024

Generating All-Atom Protein Structure from Sequence-Only Training Data

AbstractO_ST_ABSPurposeC_ST_ABSDesigning proteins with atomic-level functional control remains a central challenge in de novo design, exacerbated by limitations in structural data availability. MethodsWe introduce PLAID (Protein Latent Induced Diffusion), a generative model that efficiently co-generates discrete sequence and all-atom structure by sampling directly in the shared sequence-structure latent of a pretrained sequence-to-structure predictor. Unlike existing de novo generative models, PLAID trains its diffusion model exclusively on sequences, expanding effective training corpora by 2-4 orders of magnitude relative to structural databases. Using classifier-free guidance, PLAID supports controllable generation based on function (Gene Ontology) and organism keywords. ResultsIn silico, PLAID can unconditionally generate all-atom structures without explicit structural supervision during diffusion model training. Function conditioned proteins can recapitulates catalytic side-chain positions for residues at non-adjacent positions, and transmembrane proteins with expected hydrophobicity patterns and predicted topologies. We further experimentally validate that PLAID can be prompted to generate heme binding proteins with high sequence novelty. ConclusionOverall, PLAID unifies sequence-scale training with atomic-level generation, enabling more precise functional control in protein design. Code and weights: github.com/amyxlu/plaid.

bioinformatics↗

bioRxiv2024

ProtNote: a multimodal method for protein-function annotation

Understanding the protein sequence-function relationship is essential for advancing protein biology and engineering. However, fewer than 1% of known protein sequences have human-verified functions. While deep learning methods have demonstrated promise for protein function prediction, current models are limited to predicting only those functions on which they were trained. Here, we introduce ProtNote, a multimodal deep learning model that leverages free-form text to enable both supervised and zero-shot protein function prediction. ProtNote not only maintains near state-of-the-art performance for annotations in its train set, but also generalizes to unseen and novel functions in zero-shot test settings. We envision that ProtNote will enhance protein function discovery by enabling scientists to use free text inputs, without restriction to predefined labels - a necessary capability for navigating the dynamic landscape of protein biology.

bioengineering↗

bioRxiv2024

Mapping the combinatorial coding between olfactory receptors and perception with deep learning

The sense of smell remains poorly understood, especially in contrast to visual and auditory coding. At the core of our sense of smell is the olfactory information flow, in which odorant molecules activate a subset of our olfactory receptors and combinations of unique receptor activations code for unique odors. Understanding this relationship is crucial for unraveling the mysteries of human olfaction and its potential therapeutic applications. Despite this, predicting molecule-OR interactions remains incredibly difficult. Here, we develop a novel, biologically-inspired approach denoted MolOR that first maps odorant molecules to their respective olfactory receptor (OR) activation profiles and subsequently predicts their odor percepts. Despite a lack of overlap between molecules with OR activation data and percept annotations, our joint model improves percept prediction by leveraging the OR activation profile of each odorant as auxiliary features in predicting its percepts. We extend this cross receptor-percept approach, showing that sets of molecules with very different structures but similar percepts, a common challenge for chemosensory prediction, have similar predicted OR activation profiles. Lastly, we further probe the odorant-OR models predictive ability, showing it can distinguish binding patterns across unique OR families, as well as between protein-coding genes or frequently occuring pseudogenes in the human olfactory subgenome. This work may aid in the potential discovery of novel odorant ligands targeting functions of orphan ORs, and in further characterizing the relationship between chemical structures and percepts. In doing so, we hope to advance our understanding of olfactory perception and the design of new odorants with desired perceptual qualities.

molecular biology↗

bioRxiv2024

Tokenized and Continuous Embedding Compressions of Protein Sequence and Structure

Existing protein machine learning representations typically model either the sequence or structure distribution, with the other modality implicit. The latent space of sequence-to-structure prediction models such as ESMFold represents the joint distribution of sequence and structure; however, we find these embeddings to exhibit massive activations, whereby some channels have values 3000x higher than others, regardless of the input. Further, on continuous compression schemes, ESMFold embeddings can be reduced by a factor of 128x along the channel and 8x along the length, while retaining structure information at <2[A] scale accuracy, and performing competitively on protein function and localization benchmarks. On discrete compression schemes, we construct a tokenized all-atom structure vocabulary that retains high reconstruction accuracy, thus introducing a tokenized representation of all-atom structure that can be obtained from sequence alone. We term this series of embeddings as CHEAP (Compressed Hourglass Embedding Adaptations of Proteins) embeddings, obtained via the HPCT (Hourglass Protein Compression Transformer) architecture. CHEAP is a compact representation of both protein structure and sequence, sheds light on information content asymmetries between sequence and structure, democratizes representations captured by large models, and is designed to have flexible downstream applications such as generation, search, and prediction.

bioinformatics↗

bioRxiv2024

Feature Reuse and Scaling: Understanding Transfer Learning with Protein Language Models

Large pretrained protein language models (PLMs) have improved protein property and structure prediction from sequences via transfer learning, in which weights and representations from PLMs are repurposed for downstream tasks. Although PLMs have shown great promise, currently there is little understanding of how the features learned by pretraining relate to and are useful for downstream tasks. We perform a systematic analysis of transfer learning using PLMs, conducting 370 experiments across a comprehensive suite of factors including different downstream tasks, architectures, model sizes, model depths, and pretraining time. We observe that while almost all down-stream tasks do benefit from pretrained models compared to naive sequence representations, for the majority of tasks performance does not scale with pretraining, and instead relies on low-level features learned early in pretraining. Our results point to a mismatch between current PLM pretraining paradigms and most applications of these models, indicating a need for better pretraining methods.

bioengineering↗

bioRxiv2023

Protein generation with evolutionary diffusion: sequence is all you need

Deep generative models are increasingly powerful tools for the in silico design of novel proteins. Recently, a family of generative models called diffusion models has demonstrated the ability to generate biologically plausible proteins that are dissimilar to any actual proteins seen in nature, enabling unprecedented capability and control in de novo protein design. However, current state-of-the-art diffusion models generate protein structures, which limits the scope of their training data and restricts generations to a small and biased subset of protein design space. Here, we introduce a general-purpose diffusion framework, EvoDiff, that combines evolutionary-scale data with the distinct conditioning capabilities of diffusion models for controllable protein generation in sequence space. EvoDiff generates high-fidelity, diverse, and structurally-plausible proteins that cover natural sequence and functional space. We show experimentally that EvoDiff generations express, fold, and exhibit expected secondary structure elements. Critically, EvoDiff can generate proteins inaccessible to structure-based models, such as those with disordered regions, while maintaining the ability to design scaffolds for functional structural motifs. We validate the universality of our sequence-based formulation by experimentally characterizing intrinsically-disordered mitochondrial targeting signals, metal-binding proteins, and protein binders designed using EvoDiff. We envision that EvoDiff will expand capabilities in protein engineering beyond the structure-function paradigm toward programmable, sequence-first design.

bioengineering↗

bioRxiv2023

Benchmarking Uncertainty Quantification for Protein Engineering

Machine learning sequence-function models for proteins could enable significant ad vances in protein engineering, especially when paired with state-of-the-art methods to select new sequences for property optimization and/or model improvement. Such methods (Bayesian optimization and active learning) require calibrated estimations of model uncertainty. While studies have benchmarked a variety of deep learning uncertainty quantification (UQ) methods on standard and molecular machine-learning datasets, it is not clear if these results extend to protein datasets. In this work, we implemented a panel of deep learning UQ methods on regression tasks from the Fitness Landscape Inference for Proteins (FLIP) benchmark. We compared results across different degrees of distributional shift using metrics that assess each UQ methods accuracy, calibration, coverage, width, and rank correlation. Additionally, we compared these metrics using one-hot encoding and pretrained language model representations, and we tested the UQ methods in a retrospective active learning setting. These benchmarks enable us to provide recommendations for more effective design of biological sequences using machine learning.

bioengineering↗

bioRxiv2023

SEQUENCE VS. STRUCTURE: DELVING DEEP INTO DATA DRIVEN PROTEIN FUNCTION PREDICTION

AO_SCPLOWBSTRACTC_SCPLOWPredicting protein function is a longstanding challenge that has significant scientific implications. The success of amino acid sequence-based learning methods depends on the relationship between sequence, structure, and function. However, recent advances in AlphaFold have led to highly accurate protein structure data becoming more readily available, prompting a fundamental question: given sufficient experimental and predicted structures, should we use structure-based learning methods instead of sequence-based learning methods for predicting protein function, given the intuition that a proteins structure has a closer relationship to its function than its amino acid sequence? To answer this question, we explore several key factors that affect function prediction accuracy. Firstly, we learn protein representations using state-of-the-art graph neural networks (GNNs) and compare graph construction(GC) methods at the residue and atomic levels. Secondly, we investigate whether protein structures generated by AlphaFold are as effective as experimental structures for function prediction when protein graphs are used as input. Finally, we compare the accuracy of sequence-only, structure-only, and sequence-structure fusion-based learning methods for predicting protein function. Additionally, we make several observations, provide useful tips, and share code and datasets to encourage further research and enhance reproducibility.

bioinformatics↗

bioRxiv2023

Computational Scoring and Experimental Evaluation of Enzymes Generated by Neural Networks

In recent years, generative protein sequence models have been developed to sample novel sequences. However, predicting whether generated proteins will fold and function remains challenging. We evaluate computational metrics to assess the quality of enzyme sequences produced by three contrasting generative models: ancestral sequence reconstruction, a generative adversarial network, and a protein language model. Focusing on two enzyme families, we expressed and purified over 440 natural and generated sequences with 70-90% identity to the most similar natural sequences to benchmark computational metrics for predicting in vitro enzyme activity. Over three rounds of experiments, we developed a computational filter that improved experimental success rates by 44-100%. Surprisingly, neither sequence identity to natural sequences nor AlphaFold2 residue-confidence scores were predictive of enzyme activity. The proposed metrics and models will drive protein engineering research by serving as a benchmark for generative protein sequence models and helping to select active variants to test experimentally.

biochemistry↗