Biology subjects

Celaj, A.

Publications and source records attributed to Celaj, A..

3 recordsLinked to original sources

bioRxiv2025

FlashRNA: An Efficient Model for Regulatory Genomics

Transformer-based genomic sequence-to-function models effectively capture long-range genomic interactions but incur high computational costs due to the quadratic complexity of their self-attention layers. In this work, we introduce FlashRNA, which significantly improves computational and memory efficiency through FlashAttention, advancements in model architecture, and optimized training setup. FlashRNA achieves comparable or slightly improved predictive performance compared to similar sized Borzoi or Flashzoi models, notably without depending on pre-trained weights - a major limitation of Flashzoi. Remarkably, we trained FlashRNA from scratch in one day on a single GPU, significantly accelerating training and inference speed. These improvements can facilitate further developments in models for regulatory genomics by reducing computational cost. We demonstrate this in two downstream applications: 1) we train a large ensemble of 16 FlashRNA models and distill them into a single model to improve performance while maintaining efficiency, and 2) we fine-tune FlashRNA on three prediction tasks - ChIP-seq, RNA half-life, and translation efficiency - achieving performance matching or exceeding state-of-the-art task-specific models. Code: https://github.com/deepgenomics/flashrna

genomics↗

bioRxiv2025

Sequence based prediction of cell type specific microRNA binding and mRNA degradation for therapeutic discovery

MicroRNAs and RNA binding proteins are crucial elements of post-transcriptional gene regulation, which governs the fate of mRNA molecules in the cell. However, the landscape of these regulatory interactions, particularly across different mammalian cell types, remains underexplored. We describe REPRESS, a deep learning model that predicts cell-type-specific microRNA binding and mRNA degradation directly from RNA sequence. REPRESS was trained on AGO2-CLIP, miR-eCLIP and Degradome-Seq data profiling millions of microRNA binding and mRNA degradation sites across multiple cell types in human and mouse. It reveals biology that other state-of-the-art methods did not, such as identifying repressive non-canonical miRNA target sites and decoding the regulatory effects of sequence context and miRNA binding site multiplicity. REPRESS outperforms other advanced methods and neural architectures on a comprehensive suite of seven orthogonal tasks, including identifying genetic variants that affect microRNA binding, predicting out-of-distribution data from massively parallel reporter assays, and predicting canonical and non-canonical miRNA mediated repression. To demonstrate the general utility of REPRESS, we show that it provides insights into novel biology and the design of RNA therapeutics. Code is available at : https://github.com/deepgenomics/repress

genomics↗

bioRxiv2023

An RNA foundation model enables discovery of disease mechanisms and candidate therapeutics

Accurately modeling and predicting RNA biology has been a long-standing challenge, bearing significant clinical ramifications for variant interpretation and the formulation of tailored therapeutics. We describe a foundation model for RNA biology, "BigRNA", which was trained on thousands of genome-matched datasets to predict tissue-specific RNA expression, splicing, microRNA sites, and RNA binding protein specificity from DNA sequence. Unlike approaches that are restricted to missense variants, BigRNA can identify pathogenic non-coding variant effects across diverse mechanisms, including polyadenylation, exon skipping and intron retention. BigRNA accurately predicted the effects of steric blocking oligonucleotides (SBOs) on increasing the expression of 4 out of 4 genes, and on splicing for 18 out of 18 exons across 14 genes, including those involved in Wilson disease and spinal muscular atrophy. We anticipate that BigRNA and foundation models like it will have widespread applications in the field of personalized RNA therapeutics.

bioinformatics↗