FlashRNA

FlashRNA is an efficient sequence-to-function model for regulatory genomics, developed by Deep Genomics and released as a bioRxiv preprint in October 2025. Transformer models that predict functional genomics signals directly from DNA sequence—epitomized by Enformer and Borzoi—capture long-range regulatory interactions but are expensive to train and deploy because of the quadratic cost of self-attention over very long input windows. FlashRNA addresses this bottleneck, matching the predictive quality of comparable models while training in roughly one day on a single GPU and without depending on any pre-trained weights.

The model builds on the lineage of Borzoi and its efficiency-focused successor Flashzoi, incorporating FlashAttention together with architectural refinements to reduce compute and memory demands. This makes accurate genome-scale prediction accessible to labs without large clusters, and it lowers the barrier to iterating on and fine-tuning sequence-to-function models for new assays and organisms.

#Key Features

• FlashAttention-based efficiency: The self-attention layers use FlashAttention plus architectural enhancements, cutting the compute and memory footprint that limits conventional long-context genomic transformers.
• Trains from scratch in one day: FlashRNA reaches competitive accuracy after being trained from scratch on a single GPU in about a day, without initializing from another pre-trained genomic model.
• Ensemble distillation: Multiple model replicates are distilled into a single compact student, retaining ensemble-level performance at lower inference cost.
• Task fine-tuning: The pretrained backbone transfers to downstream tasks including ChIP-seq, RNA half-life, and translation efficiency prediction with competitive results.
• Open checkpoints: Four single-model replicates and one distilled variant are released under Apache 2.0 through Weights & Biases artifacts, loadable via a Python API.

#Technical Details

FlashRNA is a transformer that maps DNA sequence to a large panel of functional genomics tracks. It is trained on data processed and shared by the authors of Borzoi—including RNA-seq, DNase-seq, and ATAC-seq coverage—with additional preprocessing. The authors demonstrate that FlashRNA matches comparable Borzoi-class models on the sequence-to-function objective despite its substantially reduced training cost, then show transfer to three distinct prediction problems: transcription-factor binding (ChIP-seq), RNA stability (half-life), and translation efficiency. Ensemble distillation consolidates replicate models into one network for efficient deployment. The current implementation requires GPUs that support FlashAttention, with broader hardware support noted as forthcoming.

#Applications

FlashRNA is aimed at regulatory genomics researchers who need accurate predictions of how sequence shapes transcription, RNA processing, stability, and translation. Because it trains quickly and openly from scratch, it is well suited to groups that want to retrain or fine-tune on their own assays rather than rely on a fixed foundation model. Downstream uses include interpreting non-coding sequence, prioritizing regulatory variants, and modeling post-transcriptional regulation such as RNA half-life and translation efficiency—capabilities directly relevant to RNA therapeutics and functional genomics pipelines.

#Impact

FlashRNA reframes access to genome-scale sequence-to-function modeling: rather than treating large pretrained transformers as fixed and costly resources, it shows that Borzoi-class accuracy can be reached on a single GPU in about a day, from scratch. This dramatically lowers the barrier for academic and industry labs to build, adapt, and iterate on regulatory genomics models. The public Apache-2.0 code and checkpoints, along with the demonstrated fine-tuning recipes, position FlashRNA as a practical backbone for post-transcriptional and regulatory prediction tasks. As a preprint from Deep Genomics, its results await peer review, and the FlashAttention hardware requirement currently constrains deployment on some GPUs.