siDiff

A two-kilobase transcript offers roughly two thousand possible 21-nucleotide siRNA target sites, and essentially every computational siRNA pipeline handles that by enumerating all of them, scoring each with a trained efficacy model, and keeping the top few. The ranking is only as good as the predictor's reach: on a novel target, the motif preferences it absorbed from a few thousand measured duplexes quietly decide which regions of the transcript are ever considered, and potent sites outside those preferences are filtered out before scoring begins.

siDiff reverses the direction of the search: rather than scoring a fixed library, it samples duplexes from a conditional distribution over the target mRNA. The generator is a masked discrete diffusion transformer — siRNA positions start in an absorbing [MASK] state and are progressively unmasked into {A, U, G, C}, with cross-attention tying each position to the surrounding mRNA context. A separately trained efficacy predictor steers that unmasking trajectory at inference, so the diffusion prior supplies sequence diversity while the predictor keeps samples from drifting into non-functional space.

The framework comes from XtalPi with Peking University Health Science Center and Tsinghua University. It approaches from the generative side the same bottleneck that discriminative oligonucleotide models such as ASOCompass attack by ranking.

#Key Features

• Target-conditioned generation: A conditional distribution over duplexes given the mRNA replaces the enumerate-then-rank pipeline, so candidate sites are proposed rather than pre-selected.
• Efficacy-guided unmasking: At each reverse step the guidance model scores every hypothetical nucleotide at every unresolved position, and those scores feed both the choice of which position to unmask and which base to place there.
• Three-stage biology-aware sampling: Early steps reject candidates that cannot form a valid reverse-complement duplex against the target segment, middle steps apply fine-grained efficacy guidance, and a clustering stage suppresses candidates crowding one narrow window of the transcript.
• Mask-robust guidance: A three-stage curriculum trains the predictor on progressively more heavily masked sequences, so it stays reliable on the partial states the sampler actually queries.
• Generation-aware metrics: Hit_δ@K scores target-site recovery within a positional tolerance, and Efficacy-Window Coverage counts how many distinct high-efficacy regions of a transcript a method recovers — evaluation built for generators, not rankers.

#Technical Details

The generator is a six-layer CrossAttentionDiT with hidden dimension 256 and eight attention heads; each block combines self-attention over the siRNA representation, cross-attention to a 61-nt mRNA context window, and a feed-forward network under timestep modulation, totalling 7.6M trainable parameters. Training uses a masked-token reconstruction loss plus an auxiliary pairwise ranking loss at weight 0.2, run for 50 epochs on a single A100. Randomly sampling the flank offsets around each target site yields 3,860 mRNA-context–siRNA instances from Huesken-derived measurements, split 3,088 / 772. Two efficacy models support inference: a partial-state predictor over a 21-nt target segment and the current masked siRNA state (validation PCC 0.677, SCC 0.684), and a full-sequence predictor over a 31-nt context that scores completed candidates before cluster ranking.

The Takayuki benchmark and three patent-derived target datasets — ANGPTL7, CTNNB1 and GSK3A — are excluded from training entirely and used as external evaluation, so one trained model designs for all four targets without a refit. On Takayuki siDiff reaches Hit₀@1 0.242 and Hit₃@3 0.664, against 0.038 / 0.401 for enumeration ranked by iScore and 0.048 / 0.318 for enumeration ranked by OligoFormer; on CTNNB1 it reaches 0.200 / 0.654 versus 0.022 / 0.385, while recovering 29 of 29 ground-truth efficacy windows. Ablating the clustering stage costs 0.150 mean Hit₀@1 and 0.183 mean Hit₃@3, making redundancy control as load-bearing as the guidance itself.

#Applications

The intended use is early-stage RNAi therapeutic discovery: given a target transcript, produce a compact panel of candidate 21-mers spread across several high-efficacy regions rather than clustered on one site, before committing to synthesis and cell-based screening. Holding out the patent-derived targets makes that setting explicit — a new gene with no in-house efficacy data. The model designs naked siRNA sequences; chemical modification patterns, which govern stability and immunogenicity in vivo, remain a separate design step.

#Impact

siDiff argues that oligonucleotide design should be generative, and it makes that argument on the measurement that matters — recovering diverse potent regions of unseen transcripts, where rank-based pipelines are weakest. Its metrics are a contribution in their own right, capturing something the regression correlations reported by efficacy predictors do not. The caveats are real: at 7.6M parameters trained on a few thousand measured duplexes, this is a small model in a data-poor corner of the field; the patent-derived evaluation sets rely on a proxy label taken as the maximum observed efficacy across modification patterns, and are not redistributable; and all results are in silico, with no wet-lab validation of generated candidates. The work is a preprint awaiting peer review, and the repository announced in the paper currently holds only a README — no code, weights, or configs have been posted.