Biology subjects

Ko, Y. S.

Publications and source records attributed to Ko, Y. S..

4 recordsLinked to original sources

bioRxiv2024

ProteinAligner: A Multi-modal Pretraining Framework for Protein Foundation Models

Protein foundation models, particularly protein language models, have demonstrated strong success in learning meaningful representations of proteins using transformer architectures pretrained on large-scale protein datasets with self-supervised learning. These representations have been highly effective for downstream tasks such as predicting protein functions and properties. However, most current protein foundation models focus on pretraining with amino acid sequences, often neglecting additional modalities like protein structures and related literature, both of which provide valuable insights. To address this gap, we propose a multi-modal pretraining approach that integrates three key modalities - protein sequences, structures, and literature text. In our framework, the protein sequence modality serves as the anchor, with the other two modalities aligned to it, enhancing the models capacity to capture more comprehensive protein information. ProteinAligner out-performed state-of-the-art protein foundation models in predicting protein functions and properties across diverse down-stream tasks.

bioinformatics↗

bioRxiv2024

Benchmarking text-integrated protein language model embeddings and embedding fusion on diverse downstream tasks

Protein language models (pLMs) have become essential tools in computational biology, powering diverse applications from variant effect prediction to protein engineering. Central to their success is the use of pretrained embeddings-contextualized representations of amino acid sequences-which enable effective transfer learning, especially in data-scarce settings. However, recent studies have revealed that standard masked language modeling objectives often produce representations that are misaligned with the needs of downstream tasks. While scaling up model size improves performance in some cases, it does not universally yield better representations. In this study, we investigate two complementary strategies for improving pLM representations: (1) integrating text annotations through contrastive learning, and (2) combining multiple embeddings via embedding fusion. We benchmark six text-integrated pLMs (tpLMs) and three large-scale pLMs across six biologically diverse tasks, showing that no single model dominates across settings. Fusion of multiple tpLMs embeddings improves performance on most tasks but presents a computational bottleneck due to the combinatorial number of possible combinations. To overcome this, we introduce greedier forward selection, a linear-time algorithm that efficiently identifies near-optimal embedding subsets. We validate its utility through two case studies, homologous sequence recovery and protein-protein interaction prediction, demonstrating new state-of-the-art results in both. Our work highlights embedding fusion as a practical and scalable strategy for improving protein representations.

bioinformatics↗

bioRxiv2024

RESP2: An uncertainty aware multi-target multi-property optimization AI pipeline for antibody discovery

Discovery of therapeutic antibodies against infectious disease pathogens presents distinct challenges. Ideal candidates must possess not only the properties required for any therapeutic antibody (e.g. specificity, low immunogenicity) but also high affinity to many mutants of the target antigen. Here we present RESP2, an enhanced version of our RESP pipeline, designed for the discovery of antibodies against one or multiple antigens with simultaneously optimized developability properties. We first evaluate this pipeline in silico using the Absolut! database of scores for antibodies docked to target antigens. We show that RESP2 consistently identifies sequences that bind more tightly to a group of target antigens than any sequence present in the training set with success rates >= 85%. Popular generative AI techniques evaluated on the same datasets achieve success rates of 1.5% or less by comparison. Next we use the receptor binding domain (RBD) of the COVID-19 spike protein as a case study, and discover a highly human antibody with broad (mid to high-affinity) binding to at least 8 different variants of the RBD. These results illustrate the advantages of this pipeline for antibody discovery against a challenging target. A Python package that enables users to utilize the RESP pipeline on their own targets is available at https://github.com/Wang-lab-UCSD/RESP2, together with code needed to reproduce the experiments in this paper.

bioinformatics↗

bioRxiv2024

TUnA: An uncertainty aware transformer model for sequence-based protein-protein interaction prediction

Protein-protein interactions (PPIs) are important for many biological processes, but predicting them from sequence data remains challenging. Existing deep learning models often cannot generalize to proteins not present in the training set, and do not provide uncertainty estimates for their predictions. To address these limitations, we present TUnA, a Transformer-based uncertainty aware model for PPI prediction. TUnA uses ESM-2 embeddings with Transformer encoders and incorporates a Spectral-normalized Neural Gaussian Process. TUnA achieves state-of-the-art performance and, importantly, evaluates uncertainty for unseen sequences. We demonstrate that TUnAs uncertainty estimates can effectively identify the most reliable predictions, significantly reducing false positives. This capability is crucial in bridging the gap between computational predictions and experimental validation.

bioinformatics↗