RWKV-IF

RNA inverse folding — finding a nucleotide sequence that folds into a specified secondary structure — is the design step behind riboswitches, aptamers, siRNAs, and RNA nanostructures. It is also NP-hard. The classical tools, RNAinverse from the ViennaRNA package and successors such as antaRNA and SAMFEO, explore sequence space by adaptive random walks, ant colony optimization, or Boltzmann sampling; they are reliable but slow on long or deeply nested targets, and prone to settling in local optima that satisfy most base pairs but never match the target exactly.

RWKV-IF, developed at UC San Diego, reframes the problem as conditional language modeling. A target structure in dot-bracket notation and its sequence are concatenated as a single character-level token stream, and the model learns to continue the structure prompt with nucleotides autoregressively. The backbone is RWKV — specifically the Eagle/RWKV-5 architecture — an attention-free design whose time-mixing and channel-mixing blocks give it a recurrent formulation with linear rather than quadratic cost in sequence length, and constant-memory streaming inference. Two further choices distinguish it: the training set is entirely synthetic, two million random RNA sequences folded computationally rather than a curated structure database, and control is exercised at decoding time through sampling parameters and a nucleotide-composition bias.

#Key Features

• Attention-free linear-complexity backbone: RWKV's time-mixing block accumulates a decaying key-value state, giving transformer-like long-range modeling with the memory profile of an RNN — useful when the same target must be sampled thousands of times.
• Structure-masked training objective: The loss is masked over the dot-bracket portion of each pair, so the model spends its capacity learning to predict nucleotides given structure rather than reconstructing the prompt.
• Explicit G-C content control: A bias δ added to or subtracted from the G and C logits based on the running composition steers the output toward a target ratio; requested ratios of 0.25, 0.50, and 0.75 yield actual ratios of 0.2545, 0.4972, and 0.7457.
• Temperature-tunable diversity: Lowering temperature trades sequence diversity for exactness; at 0.01 the model retains a mean pairwise sequence distance near 15 while holding full-match accuracy above 0.90.

#Technical Details

The model is small: 8 RWKV layers, an embedding size of 512, a context length of 256, and a vocabulary of 9 tokens, trained for 7 epochs on a single NVIDIA A5000 with a linear warm-up and cosine decay from 1 × 10⁻⁴ to 1 × 10⁻⁵. Training data was generated by sampling two million random sequences of length 80–120 from A, U, G, and C and folding each with ViennaRNA's RNAfold to obtain a dot-bracket label; every structure in the set is unique. Because RWKV's default training routine flattens tokens into one long vector and samples arbitrary windows — which would sever structure-sequence pairs — the authors reshape the data so that sampling occurs by row.

Evaluation uses 2,082 unique structures from EternaBench-CM, disjoint from training, with one sequence generated per structure and refolded by RNAfold for comparison. RWKV-IF reaches an average character-level correctness of 0.9943, an average edit distance of 0.5696, and a full-match accuracy of 0.9097, against 0.9540 / 4.8710 / 0.3700 for RNAinverse and 0.9680 / 3.4280 / 0.4810 for antaRNA — an 83.4% reduction in edit distance and an 89.2% relative improvement in full-match rate over those baselines. Notably the training structures are harder than the test structures by average maximum depth (19.25 versus 13.96) and span (69.68 versus 45.44), which is the authors' evidence that randomly generated sequences are a viable training substrate.

#Applications

The target use is high-throughput candidate generation in RNA design workflows: synthetic biology constructs such as biosensors and regulatory circuits, therapeutic aptamers and siRNAs, and RNA nanotechnology scaffolds, where a designer needs many sequences matching a structural motif and also needs to hit a G-C window for thermodynamic stability or synthesis feasibility. Streaming inference from a small recurrent model makes large candidate pools cheap relative to iterative search, and the high full-match rate means fewer wasted attempts.

#Impact

RWKV-IF demonstrates that a linear-attention language model, trained purely on computationally folded random sequences, transfers to curated RNA structure benchmarks — a result with implications beyond inverse folding for any biomolecular design task starved of labeled structure data. Its scope is narrow and should be read that way. It operates on secondary structure in dot-bracket notation only, not on 3D backbones as tertiary inverse-folding models such as gRNAde do, and it does not handle pseudoknots, which the authors name as future work. The comparison set covers two search-based tools and no learned baselines, and RNAfold supplies both the training labels and the scoring, so the reported accuracy measures agreement with a thermodynamic folding model rather than experimental structure. The repository ships training and inference scripts but no pretrained checkpoint and carries no license file, and the preprint has not been peer reviewed.