RNA-DCGen

An encoder-only RNA language model can tell you what a sequence will do, but it cannot write one. BERT-style models such as RNA-FM, RNA-MSM, RiNALMo and BiRNA-BERT are trained to read a nucleotide sequence and predict a property — a pairwise distance map, a secondary structure, a torsion angle — and that is a one-way street. Generative RNA design has therefore been built on separate machinery: gRNAde does geometric inverse folding, RNA-FrameFlow does flow matching over 3D backbones, and each is welded to the one constraint type it was designed around.

RNA-DCGen closes that gap without training anything generative. It reframes design as a search over the 4^N sequences of length N, and uses the predictor's own input gradients to steer the search — the same Greedy Coordinate Gradient (GCG) trick that finds adversarial suffixes for chat LLMs, pointed at nucleotides instead of tokens. A random sequence is scored against the target property, the loss is backpropagated to the one-hot input to give a gradient for each of {A, G, C, U} at every position, and those gradients rank the substitutions most likely to reduce the loss. Because the loss function is arbitrary, any property the language model can predict becomes a design objective.

The framework was developed at Bangladesh University of Engineering and Technology with a collaborator at UC Riverside, posted to bioRxiv in September 2024 and presented at the Machine Learning in Structural Biology workshop at NeurIPS 2024.

#Key Features

• Dual conditioning: The search optimizes toward a target property while arbitrary sub-sequences stay frozen — an aptamer binding site, or a CRISPR guide region that must survive intact. The property constraint and the sequence constraint are enforced together.
• Two independent language models: The 117M-parameter BiRNA-BERT is fine-tuned as the fast generator; the 650M-parameter RiNALMo, differing in architecture and training corpus, is fine-tuned separately as a discriminator that scores the result, so a sequence that merely exploits the generator's blind spots does not pass.
• Property-agnostic by construction: Distance maps and secondary structures are the two demonstrated constraints, but any property with a nucleotide-level sequence-to-label dataset can be substituted by fine-tuning the RNA-LM's prediction head.
• Annealed replacement schedule: The number of positions mutated per step decays from an initial fraction toward a floor as optimization proceeds, coarse early and fine late.

#Technical Details

Each iteration computes gradients of the property loss with respect to the one-hot input, selects a random subset of modifiable positions, and draws B candidate sequences by sampling replacement nucleotides from a softmax over those gradients; the lowest-loss candidate seeds the next step. Distance-map supervision comes from the trRosettaRNA web server's dataset and secondary-structure supervision from the bpRNA split of the BEACON benchmark. Conserved regions in the experiments were simulated by fixing the three most common sub-sequences of at least five nucleotides per RNA, taken from BiRNA-BERT's tokenization statistics.

Judged by the RiNALMo discriminator, distance-map-conditioned designs reach an R² of 0.693 with conserved regions fixed and 0.625 for full generation, against 0.257 and 0.118 for random sequences of matched length and 0.810 for the natural sequences themselves. Secondary-structure conditioning gives F1 0.513 fixed and 0.400 full, against a random baseline of 0.005 and a natural-sequence ceiling of 0.729 — bounded partly by the forward task, since RiNALMo's own secondary-structure prediction scores F1 0.73. A representative 104-nucleotide design was checked independently with AlphaFold 3, reproducing the target distance map at R² 0.60. Quality falls off as sequences lengthen and the search space grows.

#Applications

The conserved-region mechanism is what makes this practical for therapeutic design: an aptamer needs a specific docking motif, a CRISPR guide RNA needs a fixed targeting region, and both must be embedded in a scaffold that folds correctly. RNA-DCGen lets a designer nail down those segments and optimize the surrounding sequence toward a desired fold, which suits the short constructs typical of RNA therapeutics and gene-editing reagents. The same recipe transfers to any lab that already has a fine-tuned property predictor and a target it wants sequences to satisfy.

#Impact

RNA-DCGen's contribution is a repurposing argument rather than a new pretrained model: the large installed base of encoder-only RNA language models can be used generatively as-is, which matters in a field where labeled structural data is too scarce to train dedicated generators for every property. The trade-offs are real. Full-sequence generation lags conserved-region generation on both tasks, longer targets degrade, and extending to a new property still requires a nucleotide-level labeled dataset and a fine-tuning run. The work remains an unreviewed preprint under an all-rights-reserved license, and it is published with no code or data availability statement — no implementation, fine-tuned checkpoints, or generated sequence sets accompany it, so reproducing the pipeline means rebuilding it.