Ghazikhani, H. | Search bioRxiv

bioRxiv2024

TooT-PLM-P2S: Incorporating secondary structure information into protein language models

In bioinformatics, modeling the protein space to better predict function and structure has benefitted from Protein Language Models (PLMs). Their basis is the proteins amino acid sequence and self-supervised learning. Ankh is a prime example of such a PLM. While there has been some recent work on integrating structure with a PLM to enhance predictive performance, to date there has been no work on integrating secondary structure rather than three-dimensional structure. Here we present TooT-PLM-P2S that begins with the Ankh model pre-trained on 45 million proteins using self-supervised learning. TooT-PLM-P2S builds upon the Ankh model by initially using its pre-trained encoder and decoder. It then undergoes an additional training phase with approximately 10,000 proteins and their corresponding secondary structures. This retraining process modifies the encoder and decoder, resulting in the creation of TooT-PLM-P2S. We then assess the impact of integrating secondary structure information into the Ankh model by comparing Ankh and TooT-PLM-P2S on eight downstream tasks including fluorescence and solubility prediction, sub-cellular localization, and membrane protein classification. For both Ankh and TooT-PLM-P2S the downstream tasks required task-specific training. Few of the results showed statistically significant differences. Ankh outperformed on three of the eight tasks, TooT-PLM-P2S did not outperform on any task for the primary metric. TooT-PLM-P2S did outperform for the precision metric for the task of discriminating membrane proteins from non-membrane proteins. This study requires future work with expanded datasets and refined integration methods.

bioinformatics↗

bioRxiv2023

Exploiting protein language models for the precise classification of ion channels and ion transporters

This study presents TooT-PLM-ionCT, a composite framework consisting of three distinct systems, each with different architectures and trained on unique datasets. Each system within TooT-PLM-ionCT is dedicated to a specific task: segregating ion channels (ICs) and ion transporters (ITs) from other membrane proteins and differentiating ICs from ITs. These systems exploit the capabilities of six diverse Protein Language Models (PLMs) - ProtBERT, ProtBERT-BFD, ESM-1b, ESM-2 (650M parameters), and ESM-2 (15B parameters). As these proteins play a pivotal role in the regulation of ion movement across cellular membranes, they are integral to numerous biological processes and overall cellular vitality. To circumvent the costly and time-consuming nature of wet lab experiments, we harness the predictive prowess of PLMs, drawing parallels with techniques in natural language processing. Our strategy engages six classifiers, embracing both conventional methodologies and a deep learning model, for each of our defined tasks. Furthermore, we delve into critical factors influencing our tasks, including the implications of dataset balancing, the effect of frozen versus fine-tuned PLM representations, and the potential variance between half and full precision floating-point computations. Our empirical results showcase superior performance in distinguishing ITs from other membrane proteins and differentiating ICs from ITs, while the task of discriminating ICs from other membrane proteins exhibits results commensurate with the current state-of-the-art. Author summaryIn our research, we have designed TooT-PLM-ionCT, a composite framework composed of three unique systems, each tailored to a specific protein classification task and trained on different datasets. This framework is our tool for categorizing integral membrane proteins, specifically ion channels and ion transporters. These proteins are essential to the health of cells, as they manage ion movement across cell membranes. To bypass the high costs and long timelines of conventional lab experiments, we have turned to advanced computation methods akin to how computers process human language. Our three-pronged approach harnesses six top-tier Protein Language Models and a range of classifiers to discern between these key proteins. In doing so, we also evaluated the effects of various conditions, like dataset balance, representation methods, and levels of computation precision, on the accuracy of our classification tasks. The outcomes show our framework effectively identifies ion transporters, sets them apart from ion channels, and distinguishes ion channels on par with existing top-notch techniques. The performance, however, can vary based on the task, suggesting that customizing the approach for each task could be beneficial. In the future, we plan to expand the depth and breadth of our protein study by incorporating additional knowledge sources, utilizing more refined representation methods, and testing our framework on larger and diverse protein datasets. This progress sets us on a path to better understand proteins and their roles in cellular health.

bioinformatics↗