codonGPT

Generative sequence models in biology have concentrated on proteins, DNA, and non-coding RNA, leaving protein-coding mRNA in an awkward gap. Coding sequences have structure none of those substrates share: they are read in non-overlapping triplets, the codon-to-amino-acid mapping is many-to-one, and the choice among synonymous codons carries organism-specific bias that shapes translation efficiency and transcript stability. A protein language model cannot see that choice at all, and a nucleotide model sees it only indirectly.

codonGPT, built by Nanil Therapeutics, is a decoder-only transformer trained from scratch on coding mRNA with a codon-level vocabulary, paired with two mechanisms that make it usable for design. The first is inference-time pre-softmax logit masking: at each step the model is restricted to codons encoding the required amino acid, turning codon design into constrained language modeling and guaranteeing the translated protein is unchanged. The second is a reinforcement learning wrapper that fine-tunes the base model per target protein against a reward combining codon adaptation index, GC balance, predicted folding free energy, rare-codon penalty, and codon-repeat penalty.

Several codon-level models already exist in the catalog — CodonFM at far larger scale, CodonTranslator for species-conditioned generation, and SynCodonLM with synonymous-constrained masked pretraining. codonGPT is the smallest of the group, and its contribution is the design loop rather than the representation: it is the first of these to apply reinforcement learning directly to a generative codon model. It appeared as a preprint in June 2025 and was published in Nucleic Acids Research.

#Key Features

• Codon-native vocabulary: 67 tokens — the 64 codons plus padding, beginning, and end markers — mean the model never has to infer reading frame, and every token boundary is a real biological unit.
• Synonymous logit masking: Restricting logits before the softmax to each position's synonymous set preserved the target protein with 100% accuracy across all 100 test genes, without touching model weights.
• Emergent codon structure: Trained with no amino acid labels, the input embeddings place synonymous codons closer together than non-synonymous pairs — mean cosine similarity 0.30 versus 0.03 — and organize by GC content and codon adaptation index along principal components.
• Per-protein reinforcement learning: The framework fine-tunes a copy of the base model for a single target gene under a user-specified multi-objective reward, rather than shipping one general optimizer.
• Extensible constraint mechanism: The same logits-processor hook that enforces synonymy can carry further inference-time constraints such as restriction site avoidance, with no retraining.

#Technical Details

The training corpus comprised 338,417 protein-coding sequences from Ensembl across seven organisms, led by human (123,845), mouse (66,558), and zebrafish (52,089) and also covering Caenorhabditis elegans, Drosophila melanogaster, Cricetulus griseus, and Escherichia coli. The released checkpoint is a GPT-2 decoder with four transformer layers, four attention heads, 256-dimensional embeddings, a 1,024-token context window, and 3.44 million parameters, which is small by foundation model standards.

Evaluation on 100 human housekeeping genes shows substantial synonymous variation with native usage respected: a median edit distance of 223 codon substitutions per gene, a median codon match rate of 39.7%, and a median cosine similarity of 0.87 (IQR 0.83–0.93) between native and generated codon frequency vectors. Generated sequences reproduce the native coupling between GC content and codon adaptation index (Pearson r = 0.58 versus 0.75), match native GC content at 53% versus 52% (p = 0.115), and shift codon adaptation index modestly upward, 0.48 versus 0.44 (p < 1 × 10⁻⁶). Case studies on HLA-A and ACTB yield reinforcement-learned variants that improve on the native sequence, the unmodified base model, and CodonTransformer, a published multispecies codon optimizer, across the reward components.

#Applications

The immediate use is codon optimization for therapeutic mRNA, recombinant protein manufacturing, and synthetic gene design, where the protein is fixed and the nucleotide sequence is the free variable. The reinforcement learning framework suits single high-value targets — a vaccine antigen, a reporter such as GFP, or a biologic like erythropoietin — because it trains a bespoke policy per gene. The base model with masking is the cheaper option for bulk generation, and the Hugging Face repository ships the custom codon tokenizer and synonymous logits processor alongside the weights so both workflows run out of the box.

#Impact

codonGPT's substantive claim is methodological: framing codon design as constrained decoding plus per-target reinforcement learning is more flexible than fine-tuning a general model on a curated dataset, because the objective can be rewritten per project without retraining the base model. That framework is more transferable than the checkpoint itself, which is orders of magnitude smaller than sibling codon models and trained on seven organisms rather than thousands of species. All evaluation is computational, with codon adaptation index, GC content, and predicted folding energy standing in for expression and stability and no wet-lab measurements reported. No training or inference source repository has been released, and the weights carry a research-use-only term rather than an open-source license, so commercial use requires contacting the developer.