Edmunds, N. S. | Search bioRxiv

bioRxiv2023

Benchmarking of AlphaFold2 accuracy self-estimates as empirical quality measures and model ranking indicators and their comparison with independent model quality assessment programs.

MotivationDespite an increase in the accuracy of predicted protein structures following the development of AlphaFold2, there remains a gap in the accuracy of predicted model quality assessment scores when compared to those generated with reference to experimental structures. The predictions of model accuracy scores generated by AlphaFold2, plDDT and pTM, have become familiar descriptors of model quality. However, at CASP15 some modelling groups noticed a variation in these scores for models of very similar observed quality, particularly for quaternary structures. There have also been a number of methods describing adaptations of the AlphaFold2 algorithm to purposes such as refinement by custom template recycling and model quality assessment using a similar method of template input. In this study we compare plDDT and pTM to their observed counterparts lDDT (including lDDT-C and lDDT-oligo) and TM-score to examine whether they retain their reliability across the whole scoring range for both tertiary and quaternary structures and in situations where the AlphaFold2 algorithm is adapted to customised functionality. In addition, we explore the accuracy with which plDDT and pTM rank AlphaFold2 tertiary and quaternary models and whether these can be improved by the independent model quality assessment programs ModFOLD9 and ModFOLDdock. ResultsFor tertiary structures it was found that plDDT was an accurate descriptor of model quality when compared to observed lDDT-C scores (Pearson {rho} = 0.97). Additionally, plDDT achieved a tertiary structure ranking agreement with observed scores of 0.34 as measured by true positive rate (TPR) and ModFOLD9 offered similar but not improved performance. However, the accuracy of plDDT (Pearson {rho} = 0.67) and pTM (Pearson {rho} = 0.70) became more variable for quaternary structures quality assessment where overprediction was seen with both scores for models of lower quality and underprediction was also seen with pTM for models of higher quality. Importantly, ModFOLDdock was able to improve upon AF2-Multimer quaternary structure model ranking as measured by both TM-score (TPR 0.34) and lDDT-oligo (TPR 0.43). Finally, evidence is presented for an increase in variability of both plDDT and pTM when custom template recycling is used, and that this variation is more pronounced for quaternary structures.

bioinformatics↗

bioRxiv2022

Improvement of protein tertiary and quaternary structure predictions using the ReFOLD4 refinement method and the AlphaFold2 recycling process

MotivationThe accuracy gap between predicted and experimental structures has been significantly reduced following the development of AlphaFold2. However, for further studies, such as drug discovery and protein design, AlphaFold2 structures need to be representative of proteins in solution, yet AlphaFold2 was trained to generate only a few structural conformations rather than a conformational landscape. In previous CASP experiments, MD simulation-based methods have been widely used to improve the accuracy of single 3D models. However, these methods are highly computationally intensive and less applicable for practical use in large-scale applications. Despite this, the refinement concept can still provide a better understanding of conformational dynamics and improve the quality of 3D models at a modest computational cost. Here, our ReFOLD4 pipeline was adopted to provide the conformational landscape of AlphaFold2 predictions while maintaining high model accuracy. In addition, the AlphaFold2 recycling process was utilised to improve 3D models by using them as custom template inputs for tertiary and quaternary structure predictions. ResultsAccording to the Molprobity score, 94% of the generated 3D models by ReFOLD4 were improved. As measured by average change in lDDT, AlphaFold2 recycling showed an improvement rate of 87.5% (using MSAs) and 81.25% (using single sequences) for monomeric AF2 models and 100% (MSA) and 97.8% (single sequence) for monomeric non-AF2 models. By the same measure, the recycling of multimeric models showed an improvement rate of as much as 80% for AF2 models and 94% for non-AF2 models. The AlphaFold2 recycling processes and ReFOLD4 method can be combined very efficiently to provide conformational landscapes at the AlphaFold2-accuracy level, while also significantly improving the global quality of 3D models for both tertiary and quaternary structures, with much less computational complexity than traditional refinement methods.

bioinformatics↗