ATGC-Gen

ATGC-Gen — Automated Transformer Generator for Controllable Generation — is a framework for designing DNA sequences that satisfy a specified biological condition: a promoter that drives a target transcription initiation profile, an enhancer active in a particular cell class, or a stretch of genome that a named transcription factor will bind in a named cell type. It was developed by Xingyu Su, Xiner Li, Yuchao Lin, and Shuiwang Ji at Texas A&M University with Ziqian Xie and Degui Zhi at UTHealth Houston, and posted as a preprint in July 2025.

The design space for conditional DNA generation has been dominated by diffusion and flow-matching methods that operate on continuous relaxations of the four-letter alphabet — the Dirichlet diffusion score model, Dirichlet flow matching, and discrete diffusion approaches such as D3. Those methods model global structure well but are not native to discrete, variable-length symbolic sequences. Language models are, and DNA language models such as DNABERT-2, Nucleotide Transformer, and HyenaDNA had largely been used to encode genomic sequence for downstream prediction rather than to generate it under control.

ATGC-Gen closes that gap with a conditioning mechanism rather than a new backbone. Heterogeneous biological signals — categorical cell-class labels, base-resolution transcription activity tracks, even the amino acid sequence of a binding protein — are projected into the model's hidden space and fused with the nucleotide embeddings, so the same architecture handles conditions of very different shape without task-specific redesign. The paper also contributes a ChIP-seq-derived generation benchmark for protein-binding specificity.

#Key Features

• Two conditioning pathways: Sequence-level integration prepends property tokens as a global prefix, suiting categorical conditions such as cell class; feature-level integration concatenates a position-aligned signal with the one-hot nucleotide at each base, suiting base-resolution tracks.
• Decoder-only and encoder-only instantiations: A GPT-style autoregressive variant supports variable-length generation; a BERT-style variant generates by masked recovery from a fully masked sequence, either in one shot or through iterative unmasking.
• Cross-modal protein conditioning: Transcription factors are represented by residue-level embeddings from ESM-2 3B rather than by an identifier, so binding conditions carry protein-sequence information.
• A ChIP-seq design benchmark: A new ENCODE-derived dataset pairs DNA with binding transcription factor and cell type, scored for functionality, fluency, and diversity.

#Technical Details

Both variants use a 768-dimensional hidden size and a single-nucleotide tokenizer: the encoder follows the bert-base configuration with 12 layers and 12 attention heads, the decoder uses 16 layers and 16 heads. Training uses AdamW at a learning rate of 1e-4 with linear warmup over the first 10% of epochs, on A100-80GB GPUs. Checkpoints are trained per task on each design dataset. The promoter task uses 100,000 GRCh38 sequences of 1,024 bp centered on annotated transcription start sites; the enhancer tasks use 104k fly-brain and 89k human-melanoma sequences of 500 bp with 81 and 47 ATAC-seq cell classes; the ChIP-seq set is filtered from roughly 10 million ENCODE rows to 55,830 examples covering 62 transcription factors, split by chromosome.

On promoter design, the BERT variant reaches an SEI-predicted regulatory-activity MSE of 0.0192, against 0.0219 for D3 and 0.0269 for Dirichlet flow matching, with the best KS statistic (0.043). On enhancer design the GPT variant roughly halves Fréchet Biological Distance versus Dirichlet flow matching — 0.508 against 1.040 on fly brain, 0.923 against 1.905 on melanoma — though with slightly lower diversity. On the ChIP-seq task, conditioning on protein and cell type lifts the binding score from 0.0747 to 0.1176, against 0.2319 for real sequences and 0.0036 for random ones. The two variants are not interchangeable: the BERT variant is far weaker on enhancers (FBD 27.63 and 45.27), where the condition is a single global label.

#Applications

The framework targets synthetic regulatory element design — promoters tuned to a desired transcription initiation profile, enhancers restricted to a target cell type, and candidate binding sites for a chosen transcription factor. These are core tasks in gene therapy vector design, cell-type-specific expression control, and synthetic biology circuit construction. The released ChIP-seq benchmark also gives method developers a conditional generation task grounded in measured protein-DNA binding rather than unconditional sequence realism.

#Impact

ATGC-Gen's contribution is to show that conditioning, not architecture, is the bottleneck in generative DNA design: a modestly sized transformer with a well-designed cross-modal interface matches or beats specialized diffusion and flow-matching generators on their own benchmarks. The paired-variant comparison is itself useful: autoregressive decoding absorbs global conditions better, while masked recovery exploits base-aligned signals better. Caveats are substantive: the work is a preprint under review, model scale was explicitly limited by compute, checkpoints are task-specific rather than a single pretrained genomic generator, and all evaluation is in silico — functionality is scored by the SEI chromatin-profile predictor and by a pretrained FBD classifier, with no wet-lab validation of the generated sequences.