EukaUTR

An AU-rich element in a 3′ untranslated region can destabilise one transcript and do nothing in another, because what the element does depends on the sequence and structure around it. That context dependence is why predictors fitted to a single reporter assay or RNA-binding protein panel transfer badly — they learn the assay, not the regulatory grammar. Sequence language models pretrained on 3′ UTRs answer this, but those preceding EukaUTR stayed inside human or mammalian sequence space, and none joined broad representation learning to both functional prediction and sequence design.

EukaUTR pretrains in two species-disjoint stages. Stage 1 learns from 7,025,323 non-redundant 3′ UTRs drawn from 1,715 genomes spanning vertebrates, non-vertebrate metazoans, plants, fungi and protists — 3.27 billion nucleotides. Stage 2 refines that checkpoint on 59,221 high-confidence 3′ UTRs from human, mouse, rat and zebrafish, four species excluded from Stage 1, so the curated corpus adds annotation quality rather than a second pass over the same sequences. Three checkpoints result: EukaUTR-S1 from Stage 1 alone, EukaUTR-S2 from full-parameter continued pretraining, and EukaUTR-S2-LoRA from low-rank adaptation.

It was posted in September 2026 by Hangzhou Institute of Medicine, Chinese Academy of Sciences with the University of Macau and the University of Chinese Academy of Sciences — the group behind CodonMamba. Where 5′ UTR-LM covers translation initiation at the 5′ end, EukaUTR is confined to the 3′ UTR.

#Key Features

• Species-disjoint two-stage pretraining: broad eukaryotic diversity first, then a curated vertebrate corpus whose four species were held out of the first stage.
• One checkpoint across thirteen tasks: fine-tuned for RBP binding, m6A sites, alternative polyadenylation, mRNA degradation and half-life, RNA abundance, translational efficiency and protein output.
• Template-free 3′ UTR generation: iterative confidence-guided masked decoding of a fully masked input at a chosen target length, with no starting sequence and no functional scorer in the loop.
• Property-directed editing: EukaUTR-Guide derives a steering direction from 100 low- and 100 high-degradation-rate UTR fragments, ranks which positions to mask with it, and lets the masked language model pick context-compatible replacements — no extra training, no candidate re-scoring.
• Regional regulatory resolution: saturation mutagenesis of the human CXCL2 3′ UTR concentrates predicted sensitivity in the 21-nt ARE1 element, not its flanks.

#Technical Details

EukaUTR is an ESM-2-style Transformer encoder — 12 layers, 12 attention heads, hidden dimension 768, about 87 million parameters — trained with a 15% masked language-modelling objective at single-nucleotide resolution over an RNA alphabet. Maximum input is 1,024 tokens (1,022 nucleotides plus <cls> and <eos>); longer 3′ UTRs are split into windows sharing a 200-nt overlap, so no position is discarded. Stage 1 ran on four NVIDIA A100s; Stage 2 either updated all weights or inserted rank-8 LoRA modules (α = 16) adding about 1.3 million trainable parameters, roughly 1.5% of the model.

Across 13 tasks from nine published datasets the checkpoints matched or beat the strongest external baseline on nearly all of them, with relative gains up to 27.45%. EukaUTR-S2-LoRA reached macro-averaged F1 0.782 on a fivefold eCLIP benchmark of 22 RBP–cell-line combinations; EukaUTR-S1 led the harder gene-held-out CLIP benchmark at F1 0.723; EukaUTR-S2 reached F1 0.968 for m6A sites across nine human cell lines, Pearson r = 0.747 on zebrafish reporter degradation rate, and r = 0.705 on protein output from a full-length human 3′ UTR MPRA. Generated sequences matched natural 3–6-mer frequencies at r = 0.941–0.981 with no pretraining-corpus near-duplicates, and EukaUTR-Guide cut the median predicted degradation rate of high-rate references by 37.2%.

#Applications

The immediate use is mRNA therapeutic and reporter design: choosing or optimising a 3′ UTR to raise transcript stability and protein output, which EukaUTR-Guide supports by editing an existing UTR rather than starting fresh. Because one encoder supplies representations for thirteen fine-tuning targets, groups with small in-house assay datasets can train a prediction head instead of a sequence model; the released pipeline covers embedding extraction, fine-tuning, generation and editing. It also serves annotation work such as scoring non-coding 3′ UTR variants.

#Impact

EukaUTR is a 3′ UTR language model pretrained across the breadth of eukaryotic sequence diversity rather than human or mammalian transcripts alone, and its two-stage design answers a common tension: broad corpora are large but noisily annotated, curated corpora clean but small. Code is MIT-licensed, all three checkpoints are downloadable, and the processed pretraining and benchmark data are on Zenodo, so the benchmarks can be rerun rather than taken on trust. Two limitations bound the results: functional evaluation concentrated on human and zebrafish data, leaving transfer to plant, fungal and protist lineages untested despite their weight in pretraining. And every generated and Guide-edited sequence was assessed computationally — against predicted degradation rates, RNAfold free energies and motif statistics — so whether the designed UTRs behave as predicted in cells remains an open experimental question, which the authors state plainly.