Loading…
RNA backbone torsion and pseudo-torsion angle prediction from sequence alone, built by fine-tuning a DNABERT checkpoint on solved RNA structures.
No providers recorded yet. Browse all providers
RNA 3D structure prediction lags well behind the protein case, partly because most methods model the problem through inter-residue distances while leaving backbone conformation implicit. An RNA backbone is defined by eight torsion angles per residue, an eight-dimensional hypertorus that Pyle and colleagues showed can be approximated by two pseudo-torsion angles. Getting those angles right is a prerequisite for reconstructing a fold, yet only one prior method — SPOT-RNA-1D, a residual CNN — predicted them from sequence alone.
RNA-TorsionBERT, from Clément Bernard, Guillaume Postic, Sahar Ghannay and Fariza Tahi at the IBISC laboratory of Université d'Évry and Université Paris-Saclay, with LISN (CNRS), uses a language model instead. Solved RNA structures are far too scarce to train a transformer from scratch, so the model starts from a checkpoint already pretrained on nucleotide sequence. Counterintuitively, the DNA model wins: the authors compare DNABERT at k = 3, 4, 5 and 6 against RNABERT, pretrained specifically on RNA, and DNABERT with 3-mers gives the best validation MCQ. They attribute this to the richer k-mer vocabulary and to capacity — the DNABERT checkpoint is around 328 MB against RNABERT's 2 MB.
A second contribution follows. Because the model predicts angles from sequence without a reference structure, its predictions can stand in for the true angles in a quality metric, yielding TB-MCQ: a reference-free scoring function that ranks candidate RNA 3D structures in torsion-angle space.
The backbone is DNABERT-3, an 86.9M-parameter BERT with a 768-dimensional hidden state and a 512-token limit; RNA sequences are handled by substituting T for U so the DNABERT tokenizer applies. On top of the last hidden state sit layer normalization, a 768→1024 linear layer, GELU, a 1024→28 linear layer and a Tanh, giving a sine and cosine for each of 14 angles per token.
Training uses 4,267 PDB structures with sequences of 11 to 508 nucleotides, filtered to below 80% sequence similarity with the evaluation sets. Validation follows SPOT-RNA-1D's 29 structures; the test set combines RNA-Puzzles and CASP-RNA into 34 structures spanning 27 to 512 nucleotides. Accuracy is reported as MCQ, a superposition-independent mean of circular quantities in degrees. RNA-TorsionBERT reaches an average MCQ of 17.4° against 19.4° for SPOT-RNA-1D, improving every individual angle, and 17.8° for angles inferred from AlphaFold 3 predictions. Broken out by RNA type the ordering shifts: for natural RNAs without homologs RNA-TorsionBERT scores 11.8° against 15.5° for AlphaFold 3, while AlphaFold 3 leads on synthetic RNAs. The advantage also grows with sequence length. As a scoring function, TB-MCQ correlates with MCQ at a Pearson coefficient of 0.87 and an enrichment score of 5.39 across three decoy sets, ahead of RASP, εSCORE, DFIRE-RNA and rsRNASP.
The direct use is supplying torsion-angle restraints to RNA 3D structure pipelines — as a starting conformation for ab initio folding or molecular dynamics relaxation, or as an input feature to a larger structure network. TB-MCQ addresses a separate need: given candidate models from any predictor, it ranks them for near-nativeness without an experimental reference, complementing distance-based scoring functions that ignore backbone conformation. Both are available through the EvryRNA platform and a Docker image, alongside the HuggingFace checkpoint.
Published in Bioinformatics, RNA-TorsionBERT is a compact and honest piece of work: it makes the case that transfer from a DNA language model beats training small on RNA, and shows that a frozen checkpoint can serve two distinct tasks. The limits are stated plainly. A training corpus of a few thousand structures is tiny by foundation-model standards, MCQ values remain high in absolute terms, and small angular deviations accumulate into large divergence, so reconstructing a full structure from predicted torsions alone is not yet viable. TB-MCQ inherits that ceiling, and is least useful for decoys very close to native. The repository carries no LICENSE file and the HuggingFace card declares its license as "other", leaving reuse terms for both code and weights undefined.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.