A fully-open structure-guided RNA foundation model for robust structural and functional inference

Abstract

RNA language models have achieved strong performances across diverse downstream tasks by leveraging large-scale sequence data. However, RNA function is fundamentally shaped by its hierarchical structure, making the integration of structural information into pre-training essential. Existing methods often depend on noisy structural annotations or introduce task-specific biases, limiting model generalizability. Here, we propose structRFM, a structure-guided RNA foundation model that is pre-trained on millions of RNA sequences and secondary structures data by integrating base pairing interactions into masked language modeling through a novel pair matching operation. We further introduce MUSES (multi-source ensemble of secondary structures) to mitigate model bias, and a dynamic masking ratio to balance the structure-guided mask and nucleotide-level mask. structRFM learns joint knowledge of sequential and structural data, producing versatile representations, including classification-level, sequence-level, and pairwise matrix features, that support a broad spectrum of downstream adaptations. structRFM ranks among the top models in zero-shot homology classification across seventeen biological language models, and sets new benchmarks for secondary structure prediction. structRFM further derives Zfold, which enables robust and reliable tertiary structure prediction, with consistent improvements in estimating 3D structures and their accordingly extracted 2D structures, achieving a pronounced about 20% performance gain compared with baselines and comparable performances with AlphaFold3 on CASP15-natural, CASP16, and RNA-Puzzles datasets. In functional tasks such as internal ribosome entry site identification, structRFM achieves a whopping 48% performance gain in F1 score. Furthermore, state-of-the-art performances in extensive experiments across novel RNA families and long non-coding RNAs indicate the robustness and generalizability of structRFM. These results demonstrate the effectiveness of structure-guided pre-training and highlight a promising direction for developing multi-modal RNA language models in computational biology. To support the broader scientific community, we have made the 21-million sequence-structure dataset and the pre-trained structRFM model fully open-source, facilitating the development of multimodal foundation models in biology.