HydraRNA

HydraRNA is an RNA language model built to handle transcripts as whole molecules rather than as fragments. Most RNA foundation models inherit the transformer's quadratic attention cost and a correspondingly short context window — RNA-FM and RiNALMo cap input at 1,024 nucleotides, RNAErnie at 512 — and were pretrained exclusively on non-coding RNA. A typical human mRNA exceeds all of those windows, so applying such models to messenger RNA means truncating or tiling the very sequence whose end-to-end organization determines its behavior.

HydraRNA was developed at the Southern University of Science and Technology in Shenzhen and published in Genome Biology in November 2025. It replaces the transformer stack with Hydra, a bidirectional state space model whose cost scales linearly with sequence length, and reinserts multi-head attention at two of twelve layers to recover the representational quality and interpretability that pure state space models give up. The result accepts sequences up to 10,000 nucleotides, and is pretrained on protein-coding mRNA as well as non-coding RNA.

A single encoder therefore covers questions that previously needed separate specialist models: 5'UTR translation efficiency, 3'UTR-driven decay, splice-site recognition, and secondary structure all become fine-tuning targets on the same backbone, and across ten downstream tasks HydraRNA outperformed the leading RNA language models on eight.

#Key Features

• Full-length transcript context: A 10,000-nucleotide input window accommodates most human mRNAs intact, so the model sees the 5'UTR, coding sequence, and 3'UTR as one interacting unit rather than as separate fragments.
• Hybrid state space and attention stack: Twelve layers of bidirectional Hydra blocks give linear-time sequence mixing, with multi-head attention substituted at layers 6 and 12 to sharpen representation quality and yield attention maps that align with base-pairing patterns.
• Coding and non-coding pretraining: Training on both mRNA and non-coding RNA yields embeddings that separate the two classes cleanly, reaching an AUC of 0.991 in supervised human coding/non-coding classification.
• Regional contribution analysis: Feeding full transcripts or isolated regions quantifies how much each part of an mRNA governs its fate — the coding sequence alone explains 42.6% of variance in translation rate and 31.4% in half-life, well ahead of either UTR.
• Mutation effect prediction: In silico mutagenesis recovers experimentally measured effects of single-base substitutions on stability and ribosome recruitment, including AU-rich element disruption in the CXCL2 3'UTR.

#Technical Details

HydraRNA has 84 million parameters across twelve layers with a hidden dimension of 1,024, and tokenizes at single-nucleotide resolution with BERT-style random masking. Pretraining used 28.09 million sequences — 13.9 million from RNAcentral release 24 and 14.2 million NCBI RefSeq transcripts spanning 1,277 species — after redundancy removal with MMseqs2 at 50% identity, totaling roughly 51 billion nucleotide tokens at a median length of 1,119 nucleotides. Sequences over 4,096 nucleotides were split during training, which took about 92 hours on eight consumer-grade RTX 4090D cards. A second checkpoint, HydraRNAv2, continues pretraining on additional non-coding RNA.

On the bpRNA TS0 secondary structure benchmark (1,305 test RNAs), HydraRNAv2 reaches an average F1 of 0.76 against 0.75 for RiNALMo, 0.68 for RNA-FM, and 0.66 for RNAErnie, despite RiNALMo carrying 650 million parameters to HydraRNA's 84 million. On 5'UTR mean ribosome load, ribosome recruitment measured by DART (Pearson R = 0.779), 3'UTR-driven stability in Beas2B cells (R = 0.694), and translation rate from pulse labeling (R² = 0.450 versus 0.376 for RiNALMo), it leads or ties the field. On RBP binding across 31 CLIP experiments it performs comparably to the specialist 3'UTRBERT (mean AUROC 0.858 versus 0.862) rather than surpassing it.

#Applications

The model suits groups engineering therapeutic mRNA and studying post-transcriptional regulation. Because it scores transcripts end to end, it can rank candidate UTR designs for translation output or stability before synthesis, and its in silico mutagenesis output localizes the specific nucleotides driving a prediction — useful for identifying regulatory elements such as AU-rich motifs or Pumilio recognition sites. Splice-site and polyadenylation-site prediction extend the same encoder to transcript annotation, with splice-site tests held out on fish, fly, plant, and worm genomes not seen during training.

#Impact

HydraRNA's contribution is architectural economy: a state space backbone that matches or beats a transformer eight times its size on RNA tasks while training in under four days on consumer hardware, lowering the barrier for labs without large GPU allocations. Its regional attribution analysis also produced a substantive biological result — a systematic quantification showing the coding sequence, not the untranslated regions, dominates variance in both mRNA translation rate and half-life. The repository carries an MIT license, but the grant is scoped to the source code: both checkpoints are distributed through a Google Drive folder rather than the repository or a versioned archive, and no terms are stated for them. Both the Genome Biology version of record and the bioRxiv preprint are CC BY-NC-ND 4.0, which restricts reuse of the published benchmark tables. The model is discriminative rather than generative, so designing new RNA sequences requires pairing it with a generative model, and the authors note that its scaling behavior beyond 84 million parameters remains untested.