DiscDiff

Diffusion models generate images by adding and removing Gaussian noise over continuous pixel values. A DNA sequence is a string over four discrete letters, where noise has no natural meaning, so generative diffusion for genomics either runs directly over categorical states, as D3PM and the Dirichlet Diffusion Score Model do, or embeds the sequence in a continuous latent space, diffuses there, and decodes back. The second route is much cheaper to train, and it fails in a specific way: the decoder must commit every latent position to one of A, C, G or T, and a single rounding error turns a TATA box into TATT — a core promoter element destroyed by one base. Those errors accumulate with sequence length, which is why latent diffusion had not been carried over to DNA.

DiscDiff is a latent diffusion model built to survive that round trip. Developed at Imperial College London by the Stan and Zhao groups with a collaborator at University College London, it pairs a two-stage variational autoencoder, which lifts one-hot DNA into an image-like latent tensor, with a U-Net denoiser trained in that latent space. Sampling is conditioned on species, so promoter-gene regions for honey bee or for human come from setting a condition rather than from refitting the model. DiscDiff first appeared in October 2023 and was extended in February 2024.

The same work introduces EPD-GenDNA, the corpus DiscDiff trains on, and Absorb-Escape, an inference-time algorithm that hands low-confidence stretches of a generated sequence to a pretrained autoregressive model — here a fine-tuned HyenaDNA — for local rewriting.

#Key Features

• Two-stage latent encoder: A multi-kernel convolutional stage turns a 4 × 2,048 one-hot sequence into a 128 × 128 image-like surface, which a second CNN stage compresses to a 16 × 16 × 32 latent.
• Species-conditioned sampling: Cross-attention layers inject species identity into the denoiser, so one checkpoint generates sequences across all 15 species in the training corpus.
• Two sequence lengths: Separate models cover 256 bp windows centred on a transcription start site, for single regulatory elements, and 2,048 bp windows spanning several elements.
• Frozen decoder at generation time: The autoencoder is trained first and its decoder is locked while the denoiser is trained, so each stage gets its own objective and hardware budget.
• Absorb-Escape refinement: Positions where the decoder's softmax coefficient falls below a threshold are regenerated by an autoregressive model, correcting single-base errors inside otherwise well-formed sequences.

#Technical Details

The denoiser is a U-Net with four down and four up blocks of eight ResNet blocks each, channel widths of 256, 256, 512 and 512, and cross-attention in the third down and second up blocks; the forward process uses 1,000 noise steps with DDPM sampling. Of three autoencoder variants compared — CNN, CNN with self-attention, and 1D-Swin — the 1D-Swin encoder reconstructs sequences most accurately (99.6% versus 99.4%), but the plain CNN encoder yields the simpler latent space and the better generations, and is the one used. Each denoiser was trained on a fixed budget of 72 GPU-hours on an A100. EPD-GenDNA is drawn from EPDnew and covers 159,125 promoters linked to 130,014 genes across 15 eukaryotic species, with expression measured over 2,713 cell types. On unconditional generation, DiscDiff reaches a Sei-based Fréchet distance of 57.4 on the 256 bp set and 45.2 on the 2,048 bp set, against 161.0 and 224.0 for the strongest D3PM baseline, with TATA-box motif correlations of 0.973 and 0.858; Absorb-Escape lowers those distances to 3.21 and 4.38. Across all 15 species in conditional generation, DiscDiff tracks initiator motifs better than HyenaDNA (0.581 versus 0.488) while HyenaDNA is stronger on the TATA box.

#Applications

Synthetic promoters and regulatory regions are the raw material of gene therapy vectors, engineered genetic circuits and heterologous protein production, where a designer wants many candidate sequences that behave like natural promoters without copying one. Because sampling is conditioned on species, a plant synthetic-biology group and a mammalian cell-line group draw from the same checkpoint, and the released weights make generation a sampling job rather than a training project. The metrics packaged with EPD-GenDNA give a shared way to score candidates in silico before committing to synthesis.

#Impact

DiscDiff is the first application of latent diffusion to DNA generation, and its contribution is showing the latent route is viable once round-trip errors are handled — by architecture, through a latent space chosen for smoothness over reconstruction accuracy, or after the fact with Absorb-Escape. The accompanying corpus and metrics give later work a benchmark that earlier single-species, few-thousand-sequence datasets could not. The evaluation is entirely computational: motif distributions, latent distances and chromatin profiles are compared against natural sequences, and no generated promoter is tested in cells, so what is established is distributional realism rather than demonstrated regulatory function.