Search bioRxiv⌕ Search

Biology subjects

Penic, R. J.

Publications and source records attributed to Penic, R. J..

2 recordsLinked to original sources

B-RANKO: An ncRNA Foundation Model with Bidirectional Decoder-Only Architecture

Ribonucleic acid (RNA) plays central roles in gene expression, regulation and catalysis, and has emerged as an important target for therapeutic design. While recent RNA foundation models have achieved strong performance on representation learning tasks, relatively few approaches support both native sequence generation and transferable representation learning within a single framework. We present B-RANKO, a 150M-parameter bidirectional decoder-only ncRNA foundation model that introduces a novel bidirectional autoregressive attention mechanism, enabling a single architecture to support both de novo sequence generation and representation learning. Pretrained on a large corpus derived from RNAcentral, Rfam and GARNET, B-RANKO generates biologically plausible and thermodynamically realistic RNA sequences that exhibit strong novelty relative to their closest matches in the pretraining corpus. We further show that B-RANKO can be efficiently fine-tuned for aptamer generation and achieves competitive performance on BEACON, with particularly strong results on RNA engineering tasks. B-RANKO is publicly available on GitHub at filipvucenik/B-RANKO, together with usage examples and instructions for downloading model weights and datasets.

molecular biology↗

RNA-xLSTM: Evaluating xLSTM as an Alternative Foundation to Transformers in RNA Modeling

Transformer-based architectures currently achieve state-of-the-art performance across a wide range of domains, including biological sequence modeling. Motivated by the recent introduction of the xLSTM architecture, we investigate its effectiveness for RNA sequence modeling by comparing a 33.7M-parameter RNA-xLSTM model against two leading RNA language models: RNA-FM and RiNALMo-33M. We pretrain RNA-xLSTM on the RNAcentral database and evaluate its performance on two downstream tasks: RNA secondary structure prediction and splice site prediction. Our results show that while RNA-xLSTM underperforms compared to the similarly sized RiNALMo, it does outperform the larger RNA-FM model on certain tasks. However, its overall performance remains inconsistent, and its advantages over transformer-based models are unclear, suggesting that further work is needed to assess its true potential in RNA modeling.

bioinformatics↗