Biology subjects

Gal, Y.

Publications and source records attributed to Gal, Y..

9 recordsLinked to original sources

bioRxiv2025

Sampling Protein Language Models for Functional Protein Design

Protein language models have emerged as powerful tools for learning rich protein representations, improving performance in tasks like structure prediction, mutation effect estimation, and homology detection. Their ability to model complex sequence distributions also holds promise for designing novel, functional proteins with broad applications in therapeutics, materials, and sustainability. However, due to the vastness of sequence space, efficient exploration methods are essential for protein engineering. Despite this, most existing protein design approaches using protein language models rely on single-mutant sampling strategies borrowed from Natural Language Processing, which fail to capture critical epistatic interactions between amino acid positions that are essential for protein function. In this work, we develop a comprehensive in silico protein design evaluation framework to systematically compare different sampling methods. After a thorough review of existing sampling strategies for language models, we introduce several approaches specifically tailored for protein design. We demonstrate that sampling strategies that consider multiple mutations simultaneously significantly outperform single-mutant approaches by better capturing epistatic effects between residue pairs. We evaluated these strategies using our framework, investigating the effects of key hyperparameters and providing practical guidance on the relative strengths of each method depending on design objectives.

genetics↗

bioRxiv2023

ProteinGym: Large-Scale Benchmarks for Protein Design and Fitness Prediction

Predicting the effects of mutations in proteins is critical to many applications, from understanding genetic disease to designing novel proteins that can address our most pressing challenges in climate, agriculture and healthcare. Despite a surge in machine learning-based protein models to tackle these questions, an assessment of their respective benefits is challenging due to the use of distinct, often contrived, experimental datasets, and the variable performance of models across different protein families. Addressing these challenges requires scale. To that end we introduce ProteinGym, a large-scale and holistic set of benchmarks specifically designed for protein fitness prediction and design. It encompasses both a broad collection of over 250 standardized deep mutational scanning assays, spanning millions of mutated sequences, as well as curated clinical datasets providing high-quality expert annotations about mutation effects. We devise a robust evaluation framework that combines metrics for both fitness prediction and design, factors in known limitations of the underlying experimental methods, and covers both zero-shot and supervised settings. We report the performance of a diverse set of over 70 high-performing models from various subfields (eg., alignment-based, inverse folding) into a unified benchmark suite. We open source the corresponding codebase, datasets, MSAs, structures, model predictions and develop a user-friendly website that facilitates data access and analysis.

synthetic biology↗

bioRxiv2023

ProteinNPT: Improving Protein Property Predictionand Design with Non-Parametric Transformers

Protein design holds immense potential for optimizing naturally occurring proteins, with broad applications in drug discovery, material design, and sustainability. How-ever, computational methods for protein engineering are confronted with significant challenges, such as an expansive design space, sparse functional regions, and a scarcity of available labels. These issues are further exacerbated in practice by the fact most real-life design scenarios necessitate the simultaneous optimization of multiple properties. In this work, we introduce ProteinNPT, a non-parametric trans-former variant tailored to protein sequences and particularly suited to label-scarce and multi-task learning settings. We first focus on the supervised fitness prediction setting and develop several cross-validation schemes which support robust perfor-mance assessment. We subsequently reimplement prior top-performing baselines, introduce several extensions of these baselines by integrating diverse branches of the protein engineering literature, and demonstrate that ProteinNPT consistently outperforms all of them across a diverse set of protein property prediction tasks. Finally, we demonstrate the value of our approach for iterative protein design across extensive in silico Bayesian optimization and conditional sampling experiments.

synthetic biology↗

bioRxiv2022

TranceptEVE: Combining Family-specific and Family-agnostic Models of Protein Sequences for Improved Fitness Prediction

Modeling the fitness landscape of protein sequences has historically relied on training models on family-specific sets of homologous sequences called Multiple Sequence Alignments. Many proteins are however difficult to align or have shallow alignments which limits the potential scope of alignment-based methods. Not subject to these limitations, large protein language models trained on non-aligned sequences across protein families have achieved increasingly high predictive performance - but have not yet fully bridged the gap with their alignment-based counterparts. In this work, we introduce TranceptEVE - a hybrid method between family-specific and family-agnostic models that seeks to build on the relative strengths from each approach. Our method gracefully adapts to the depth of the alignment, fully relying on its autoregressive transformer when dealing with shallow alignments and leaning more heavily on the family-specific models for proteins with deeper alignments. Besides its broader application scope, it achieves state-of-the-art performance for mutation effects prediction, both in terms of correlation with experimental assays and with clinical annotations from ClinVar.

genetics↗

bioRxiv2022

Learning from pre-pandemic data to forecast viral antibody escape

Effective pandemic preparedness relies on anticipating viral mutations that are able to evade host immune responses in order to facilitate vaccine and therapeutic design. However, current strategies for viral evolution prediction are not available early in a pandemic - experimental approaches require host polyclonal antibodies to test against and existing computational methods draw heavily from current strain prevalence to make reliable predictions of variants of concern. To address this, we developed EVEscape, a generalizable, modular framework that combines fitness predictions from a deep learning model of historical sequences with biophysical structural information. EVEscape quantifies the viral escape potential of mutations at scale and has the advantage of being applicable before surveillance sequencing, experimental scans, or 3D structures of antibody complexes are available. We demonstrate that EVEscape, trained on sequences available prior to 2020, is as accurate as high-throughput experimental scans at anticipating pandemic variation for SARS-CoV-2 and is generalizable to other viruses including Influenza, HIV, and understudied viruses with pandemic potential such as Lassa and Nipah. We provide continually updated escape scores for all current strains of SARS-CoV-2 and predict likely additional mutations to forecast emerging strains as a tool for ongoing vaccine development (evescape.org).

immunology↗

bioRxiv2022

Mixtures of large-scale dynamic functional brain network modes

Accurate temporal modelling of functional brain networks is essential in the quest for understanding how such networks facilitate cognition. Researchers are beginning to adopt time-varying analyses for electrophysiological data that capture highly dynamic processes on the order of milliseconds. Typically, these approaches, such as clustering of functional connectivity profiles and Hidden Markov Modelling (HMM), assume mutual exclusivity of networks over time. Whilst a powerful constraint, this assumption may be compromising the ability of these approaches to describe the data effectively. Here, we propose a new generative model for functional connectivity as a time-varying linear mixture of spatially distributed statistical "modes". The temporal evolution of this mixture is governed by a recurrent neural network, which enables the model to generate data with a rich temporal structure. We use a Bayesian framework known as amortised variational inference to learn model parameters from observed data. We call the approach DyNeMo (for Dynamic Network Modes), and show using simulations it outperforms the HMM when the assumption of mutual exclusivity is violated. In resting-state MEG, DyNeMo reveals a mixture of modes that activate on fast time scales of 100-150 ms, which is similar to state lifetimes found using an HMM. In task MEG data, DyNeMo finds modes with plausible, task-dependent evoked responses without any knowledge of the task timings. Overall, DyNeMo provides decompositions that are an approximate remapping of the HMMs while showing improvements in overall explanatory power. However, the magnitude of the improvements suggests that the HMMs assumption of mutual exclusivity can be reasonable in practice. Nonetheless, DyNeMo provides a flexible framework for implementing and assessing future modelling developments.

neuroscience↗

bioRxiv2020

Large-scale clinical interpretation of genetic variants using evolutionary data and deep learning

Quantifying the pathogenicity of protein variants in human disease-related genes would have a profound impact on clinical decisions, yet the overwhelming majority (over 98%) of these variants still have unknown consequences1-3. In principle, computational methods could support the large-scale interpretation of genetic variants. However, prior methods4-7 have relied on training machine learning models on available clinical labels. Since these labels are sparse, biased, and of variable quality, the resulting models have been considered insufficiently reliable8. By contrast, our approach leverages deep generative models to predict the clinical significance of protein variants without relying on labels. The natural distribution of protein sequences we observe across organisms is the result of billions of evolutionary experiments9,10. By modeling that distribution, we implicitly capture constraints on the protein sequences that maintain fitness. Our model EVE (Evolutionary model of Variant Effect) not only outperforms computational approaches that rely on labelled data, but also performs on par, if not better than, high-throughput assays which are increasingly used as strong evidence for variant classification11-23. After thorough validation on clinical labels, we predict the pathogenicity of 11 million variants across 1,081 disease genes, and assign high-confidence reclassification for 72k Variants of Unknown Significance8. Our work suggests that models of evolutionary information can provide a strong source of independent evidence for variant interpretation and that the approach will be widely useful in research and clinical settings.

genetics↗

bioRxiv2019

Spatiotemporal Feature Selection Improves Prediction Accuracy of Multi-Voxel Pattern Classification

The importance of spatiotemporal feature selection in fMRI decoding studies has not been studied exhaustively. Temporal embedding of features allows the incorporation of brain activity dynamics into multivariate pattern classification, and may provide enriched information about stimulus-specific response patterns and potentially improve prediction accuracy. This study investigates the possibility of enhancing the classification performance by exploring spatial and temporal (spatiotemporal) domain, to identify the optimum combination of the spatiotemporal features based on the classification performance. We investigated the importance of spatiotemporal feature selection using a slow event-related design adapted from the classic Haxby et al. (2001) study. Data were collected using a multiband fMRI sequence with temporal resolution of 0.568 seconds. A wide range of spatiotemporal observations was created as various combinations of spatiotemporal features. Using both random forest, and support vector machine, classifiers, prediction accuracies for these combinations were then compared with the single time-point spatial multivariate pattern approach that uses only a single temporal observation. The results showed that on average spatiotemporal feature selection improved prediction accuracy. Moreover, the random forest algorithm outperformed the support vector machine and benefitted from temporal information to a greater extent. As expected, the most influential temporal durations were found to be around the peak of the hemodynamic response function, a few seconds after the stimuli onset until [~]4 seconds after the peak of the hemodynamic response function. The superiority of spatiotemporal feature selection over single time-point spatial approaches invites future work to design systematic and optimal approaches to the incorporation of spatiotemporal dependencies into feature selection for decoding.\n\nHighlightsO_LISpatiotemporal feature selection effect on MVPC was assessed in slow event-related fMRI\nC_LIO_LISpatiotemporal feature selection improved brain decoding accuracy\nC_LIO_LIFrom [~]2-11 seconds after stimuli onset were the most informative part of each trial\nC_LIO_LIRandom forest outperformed support vector machines\nC_LIO_LIRandom forest benefited more from temporal changes compared with support vector machine\nC_LI

neuroscience↗

bioRxiv2019

EEG-based Prediction of Cognitive Load in Intelligence Tests

Measuring and assessing the cognitive load associated with different tasks is crucial for many applications, from the design of instructional materials to monitoring the mental wellbeing of aircraft pilots. The goal of this paper is to utilize EEG to infer the cognitive workload of subjects during intelligence tests. We chose the well established advanced progressive matrices test, an ideal work-frame because it presents problems at increasing levels of difficulty, and has been rigorously validated in past experiments. We train classic machine learning models using basic EEG measures as well as measures of network connectivity and signal complexity. Our findings demonstrate that cognitive load can be well predicted using these features, even for a low number of channels. We show that by creating an individually tuned neural network for each subject, we can improve prediction compared to a general model and that such models are robust to decreasing the number of available channels as well.

neuroscience↗