RESM

RNA language models have a data problem that protein language models do not. A four-letter alphabet carries about 2 bits per position against roughly 4.3 bits for the twenty amino acids, experimentally solved RNA structures make up about 3% of the Protein Data Bank, and RNA families are shallow — Rfam catalogs roughly 4,000 families with a median of only about 45 homologs each. Benchmarking has repeatedly shown the consequence: RNA language models tend to specialize, doing well on secondary structure or on sequence classification but rarely both.

RESM (RNA Evolutionary-Scale Model) attacks this by borrowing a protein model's learned priors outright. RNA sequences are mapped onto a pseudo-protein alphabet — A, U, C, G rewritten as K, D, N, Y — which lets every parameter of ESM-2 be updated during transfer training rather than freezing the backbone or adapting selected layers, as earlier protein-to-RNA transfer attempts did. Training then proceeds on 23.51 million de-redundant noncoding RNA sequences curated from RNAcentral.

The model was developed by a collaboration led by Yikun Zhang and Yaoqi Zhou at Shenzhen Bay Laboratory, with Peking University, the China Mobile Research Institute, Huawei, ShanghaiTech University, and Guangzhou National Laboratory, and posted as a preprint in August 2025.

#Key Features

• Pseudo-protein alphabet mapping: Recoding nucleotides into a subset of the amino acid alphabet expands the information-theoretic capacity per position and makes the pretrained protein model's weights a usable starting point for full-parameter transfer training, rather than a frozen feature extractor.
• Breaks the specialization trade-off: In zero-shot evaluation across sequence classification, secondary structure, and RNA-RNA interaction prediction, RESM outperforms 12 existing RNA language models — including models that individually lead on one task but lag badly on the others.
• Generalizes past its training length: Trained on sequences capped at 1,024 nucleotides, the supervised base-pair model extends to RNAs of about 4,000 nucleotides, a limit imposed by GPU memory rather than architecture, improving on the best competing method by 81.3% in mean F1 while running over 1,000 times faster than MSA-based approaches.
• Transfers to coding RNA tasks: Despite training exclusively on noncoding RNA, fine-tuned variants lead on mRNA ribosome loading efficiency and gene expression prediction, plus RNA accessibility and siRNA activity.
• Two released checkpoints: RESM-150M (30 layers, 640-dimensional embeddings) and RESM-650M (33 layers, 1,280-dimensional embeddings) are both distributed, with the smaller model sometimes stronger on harder, low-homology benchmarks.

#Technical Details

RESM inherits the ESM-2 transformer encoder architecture and masked language modeling objective wholesale; the novelty is the input transformation and the decision to update all parameters. The AUCG-to-KDNY mapping was selected empirically against alternatives including AUCG-to-SDYR, AUCG-to-KEDR, and AUCG-to-VLER, chosen on diagnostic F1 scores for base-pair recovery. A scaling sweep across 8M, 35M, 150M, and 650M parameters showed the largest gain from 8M to 35M and the smallest from 150M to 650M, and the authors capped the study at 650M on cost grounds. Training ran on Huawei Ascend hardware. Benchmarks span Rfam and ArchiveII classification, bpRNA-1m and TS70 base pairing — where RESM-650M reaches a median F1 of 0.637 in zero-shot secondary structure prediction and supervised RESM-basepair variants reach median F1 near 0.75 — and comparisons against 23 non-language-model secondary structure methods and RNA-specific models including RNA-FM, RNA-MSM, UNI-RNA, and AIDO.RNA.

#Applications

The model serves as a general-purpose RNA sequence encoder: embeddings feed secondary structure prediction for long transcripts that defeat covariance-model homology search, RNA-RNA interaction and target prediction, and functional family classification. The demonstrated fine-tuning tasks point at therapeutic and synthetic biology workflows — 5' UTR design guided by ribosome loading predictions, expression-level estimation across cell types, and siRNA activity ranking. The speed advantage over MSA-based methods makes transcriptome-scale annotation practical rather than aspirational.

#Impact

RESM's central claim is methodological: rather than training an RNA model from scratch on a data-poor domain, recode the substrate so that a well-trained protein model's representations can be inherited and refined. That result reframes the alphabet mismatch between nucleic acids and proteins as an engineering choice rather than a hard barrier, and it offers a template other data-limited biological sequence domains could follow. Source code and inference scripts are released under the MIT license, with pretrained weights for both checkpoints and the pretraining and downstream datasets deposited on Zenodo. Two caveats belong on the record: the work remains a preprint that has not been peer reviewed, and the authors disclose that patent applications relating to RESM and its downstream tasks were filed by the China Mobile Research Institute and Shenzhen Bay Laboratory, with two senior authors serving as CEO and scientific advisory board chair of the RNA therapeutics company Ribopeutic.