Geometry-based BERT: an experimentally validated deep learning model for molecular property prediction in drug discovery

Abstract

Various deep learning based methods have significantly impacted the realm of drug discovery. The development of deep learning methods for identifying novel structural types of active compounds has become an urgent challenge. In this paper, we introduce a self-supervised representation learning framework, i.e., GEO-BERT. GEO-BERT considers the information of atoms and chemical bonds in chemical structures as the input, and integrates the positional information of the three-dimensional conformation of the molecule for training. Specifically, GEO-BERT enhances its ability to characterize molecular structures by introducing three different positional relationships: atom-atom, bond-bond, and atom-bond. By benchmarking study, GEO-BERT has demonstrated optimal performance on multiple benchmarks. We also performed prospective study to validate the GEO-BERT model, with screening for DYRK1A inhibitors as a case. Two potent and novel DYRK1A inhibitors (IC50: <1 M) were ultimately discovered at a hit rate of 10%. Taken together, we have developed the Geometry-based BERT model for molecular property prediction and proved its practical utility in early-stage drug discovery. Graphical Abstract GEO-BERT is a model pretrained on large-scale drug molecule data, while improving the accuracy of property prediction by using three-dimensional structural information within the molecule. O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=113 SRC="FIGDIR/small/630211v1_ufig1.gif" ALT="Figure 1"> View larger version (24K): org.highwire.dtl.DTLVardef@1e9bc54org.highwire.dtl.DTLVardef@4b6832org.highwire.dtl.DTLVardef@a5d8eeorg.highwire.dtl.DTLVardef@44f7be_HPS_FORMAT_FIGEXP M_FIG C_FIG