Non-coding RNA generative model using masked discrete diffusion to sample and inpaint sequences conditioned on RNA type, trained on 4.6M ncRNAs.
No providers recorded yet. Browse all providers
An RNA designer rarely starts from nothing. More often a catalytic core, a seed region or a stem must stay exactly where it is while the rest of the sequence is rebuilt around it. Autoregressive RNA generators are a poor fit for that job: they emit nucleotides left to right, so an early position is chosen without seeing the partner it will have to base-pair with downstream. Continuous diffusion models have the opposite problem — they move between sequence space and a latent space, and hard sequence constraints are awkward to impose across that conversion.
RNA-MDLM, from Karlsruhe Institute of Technology with Forschungszentrum Jülich and the Institute of Science Tokyo, takes a third route: masked discrete diffusion, in which the forward process progressively replaces nucleotides with a mask token and the network learns to reverse that corruption with bidirectional attention. Generation starts from a fully masked sequence, inpainting from a partly masked one, and both use the same trained network, so holding a motif fixed requires no retraining — the known positions are simply never masked. The preprint was posted in September 2026.
Two additions distinguish it from EvoFlow-RNA, the other masked discrete diffusion model for non-coding RNA. The RNA class is supplied as a frozen embedding from NucleicBERT, the same group's RNA language model, rather than as an integer label; and a modified classifier-free guidance scheme (Mod-CFG) contrasts the conditional prediction against two references instead of one, which matters because RNAcentral is dominated by tRNA and rRNA.
The denoiser is a Diffusion Transformer with 12 blocks, hidden dimension 768, 12 attention heads and a conditioning dimension of 768, about 130M parameters, using adaptive layer normalisation, flash attention and rotary position embeddings. Sequences are tokenised at single-nucleotide resolution over a 12-token vocabulary and trained with the SUBS objective under a log-linear noise schedule. Training data is 4,628,983 RNAcentral sequences of length 20–512 nt annotated with RNA type; the six evaluable classes range from 2,445,367 tRNA sequences down to 7,173 snRNA. GC content and k-mer Jensen–Shannon divergence track the held-out split closely while mean identity to natural sequences stays at 61–80%, and minimum free energy per nucleotide matches validation sequences for tRNA (−0.337 vs −0.369) and snoRNA (−0.213 vs −0.215). Pooled across REPAINT GAMES, the NucleicBERT-conditioned variant recovers 0.53 of masked positions against 0.47 for EvoFlow-RNA; under a wrong class label its recovery collapses from 0.69 to 0.25, where a plain-label baseline barely moves. Against RfamGen, a per-family VAE, RNA-MDLM reaches the glmS ribozyme membership band for 75.8% of samples at a median identity of 0.64 to real members.
Conditional sampling plus inpainting fits RNA-engineering workflows where part of a molecule is fixed by function and the rest is open: transplanting a ribozyme catalytic core into a new scaffold, rebuilding a miRNA precursor stem around a fixed seed region, or generating type-specific candidate libraries to screen by folding energy before synthesis. Because a run yields a batch of a chosen type at a chosen length, the output feeds directly into secondary-structure screening pipelines.
RNA-MDLM's more durable contributions are methodological. Its ablations show that a model trained on a smaller class-balanced subset scores better on structural realism largely because 77% of its rRNA outputs are near-duplicates of abundant training sequences, while clustering those duplicates out removes memorisation at the cost of realism and inpainting accuracy — an argument that any generative RNA model reporting fidelity should report novelty alongside it. The paper also argues that class-conditioned generalists and per-family specialists solve different tasks and should not share a leaderboard. The work is a preprint awaiting peer review; evaluation is in silico throughout, resting on ViennaRNA predictions that omit pseudoknots and tertiary contacts, and no generated sequence was synthesised or assayed. The authors state that the code, trained models and benchmark will be released, but neither code nor weights are available at the time of writing.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.