PromoGen

Prokaryotic promoters look interchangeable until you try to move one. The −35 and −10 elements that RNA polymerase recognizes are species-dependent, drifting with the host's sigma factor repertoire, so a promoter tuned in Escherichia coli rarely reproduces its strength elsewhere. That would be manageable if every species had a promoter dataset to train on, but an industrially interesting Bacillus or Haloferax strain may have only a few hundred characterized promoters — far too few to fit a generative model from scratch, which is why earlier GAN-based designers were built for one organism at a time.

PromoGen answers this with transfer learning rather than more data. A single base model, PromoGen-pre, is pretrained autoregressively across many species at once, and 27 species-specific generators are then fine-tuned from it on whatever small promoter set each species actually has. The pretrained base carries the shared grammar of the promoter core region; fine-tuning supplies only the species-specific preference on top. The same pretrained checkpoint doubles as a scorer: its per-sequence log-likelihood separates promoters from non-promoters with no labeled activity data at all.

PromoGen was developed by Yi-Xin Huo's group at the Beijing Institute of Technology and published in Nucleic Acids Research in 2024. It is the predecessor of PromoGen2, which replaces the curated-promoter pretraining corpus with genome-mined upstream regions and extends coverage far beyond 27 species.

#Key Features

• Transfer learning from a few hundred sequences: PromoGen-bsu was fine-tuned on just 691 Bacillus subtilis promoters and reproduced native 6-mer frequencies at Pearson 0.900, against 0.508 for the un-fine-tuned base.
• Zero-shot promoter discrimination: log-likelihood scored off the frozen base gives an auROC of 0.60 — on par with the 2.5B-parameter Nucleotide Transformer and ahead of its 50M through 500M variants, from a model 6 to 330 times smaller.
• Character-level DNA tokenization: a 7-token vocabulary of A/T/C/G plus [PAD], [MASK] and [UNK], chosen over k-mer or byte-pair encoding so each base keeps its true position relative to the transcription start site — the offset that defines a −10 box.
• Twenty-seven species generators behind a web server: the fine-tuned checkpoints are served for on-demand generation with user-set sequence count, length, repetition penalty, top-p and temperature, plus a CNN activity filter for E. coli candidates.
• Experimentally characterized designs: 22 of 50 generated B. subtilis promoters were assayed by sfGFP fluorescence, almost all were active against random-sequence negative controls, and four matched native strong promoters.

#Technical Details

PromoGen-pre is a decoder-only transformer built on the GPT-2 recipe: six GPT-2 blocks with multi-head self-attention, a 320-dimensional hidden state, a learnable positional embedding, a feed-forward layer with ×2 expansion and GELU activation, and a maximum sequence length of 128 nucleotides, generating 5'→3' one base at a time. It totals 7.4 million parameters. Pretraining used the Prokaryotic Promoter Database — assembled from DBTBS, RegulonDB, Pro54DB and the literature — contributing more than 129,000 experimentally verified promoter and transcription-start-site sequences, about 10 million tokens, across 74 species; 27 strains hold over 500 promoters each and account for 99% of the database. Optimization was Adam at batch size 64, with fine-tuning initialized from the base weights at batch size 32 and learning rate 1e-5.

Evaluation is largely distributional. For E. coli, 200 sequences each from PromoGen-eco and a WGAN-GP baseline were scored by a Bayesian Ridge regressor selected from 12 candidate predictors; PromoGen-eco scored significantly higher, and its 6-mer frequencies tracked native promoters at Pearson 0.954 and Spearman 0.965. Across the full set of species models, fine-tuning lifted 6-mer correlation in every case, with 68% exceeding Pearson 0.5 and 32% exceeding 0.8. BLAST of the validated B. subtilis designs against the host genome found no similarity to native sequences, ruling out memorization.

#Applications

The intended user is a metabolic engineer working in an organism nobody has built a promoter library for. Selecting a species on the web server yields ranked candidates that can go straight into plasmid construction, and the spread of measured strengths — from barely active to 91% above the native P43 promoter — supports tuning pathway flux rather than only maximizing it. Because the model assigns a likelihood to any sequence, existing candidates can also be ranked before synthesis.

#Impact

PromoGen showed that a 7.4M-parameter model trained on a small, task-matched corpus can match nucleotide language models three orders of magnitude larger on a promoter task, and that pretraining across species is what makes single-species design tractable under data scarcity. Its limitations are real: distribution is through the hosted web server alone, with no released weights, training code or stated checkpoint license, so the model cannot be run locally, audited or extended to a species outside the 27; the wet-lab evidence rests on 22 constructs, characterized in B. subtilis and re-tested in E. coli; and strength cannot be specified at generation time, only filtered afterward. The successor line addresses the coverage limit directly.