TransCodon

Expressing a gene in a foreign host often fails for reasons unrelated to the protein it encodes. Synonymous codons are used at very different frequencies across species, and the standard response — rewriting a gene to use the most frequent codon at every position, maximizing the codon adaptation index (CAI) — routinely overshoots. Natural genes are not CAI-maximal. They retain rare codons where slower translation appears to help co-translational folding, and sequences engineered past that point can provoke ribosome stalling, misfolding, and host toxicity.

TransCodon, developed at the Shenzhen Institutes of Advanced Technology, CAS with collaborators at Zelixir Biotech, Shandong University, and Macao Polytechnic University, adopts a different target: generate coding sequences whose statistics resemble native genes of the chosen host rather than an idealized frequency table. It is an encoder-only transformer trained by masked language modeling over nucleotides, conditioned on an explicit species embedding, and it sees more of the transcript than codon models typically do — the 5' untranslated region alongside the coding sequence — while predicting RNA secondary structure as a second, supervised objective.

Several codon language models already occupy this space: CodonFM at far larger scale, CodonTranslator for lineage-conditioned generation, and SynCodonLM with synonymous-constrained masking. TransCodon's distinguishing choices are its nucleotide-level vocabulary, the inclusion of regulatory 5'UTR context, and an evaluation philosophy that scores proximity to natural sequences instead of CAI. The work was posted to bioRxiv in July 2025 and has not yet been peer reviewed.

#Key Features

• Native-like rather than CAI-maximal design: The model is evaluated on how closely generated sequences match natural distributions of codon similarity index, GC content, and folding free energy, deliberately avoiding the frequency-saturating behavior of earlier optimizers.
• Joint 5'UTR and CDS modeling: A separator token distinguishes the 100 nucleotides upstream of the start codon from the coding region, letting translation-initiation context inform codon choice.
• Explicit species conditioning: Each of 1,436 training species maps to a learnable token-type embedding, so users direct optimization toward a host by selecting its identifier at inference.
• RNA secondary structure supervision: Dot-bracket annotations from ViennaRNA serve as auxiliary labels, added to the masked-nucleotide loss with a weight of 0.4.
• Zero-shot fitness scoring: The likelihood a native CDS receives under the model correlates with measured protein abundance across six species, without task-specific training.

#Technical Details

The architecture follows the ESM family: 12 transformer encoder layers, 768-dimensional embeddings, 12 attention heads, a 3,072-dimensional feedforward block, rotary positional embeddings, and pre-layer normalization, with sequences up to 2,048 tokens. The vocabulary is only 10 tokens — A, T, C, G plus special symbols — finer-grained than the amino-acid/codon pair tokens used by CodonTransformer, allowing partial decoding when an amino acid is known but its codon is not. Pretraining covered roughly 5.5 million coding sequences from 1,436 species (41.4% bacteria, 25.9% archaea, 32.8% eukaryotes) drawn from four datasets, two of which include 5'UTRs; 25% of tokens were masked per batch over five epochs on four V100 GPUs.

On a CD-HIT-filtered holdout of about 6,400 sequences from eight species, TransCodon recovers 49.0% of native codons, matching fine-tuned CodonTransformer (48.6%) without fine-tuning. Its generated sequences track natural distributions far more closely: KL divergence on codon similarity index in E. coli is 3.31 versus CodonTransformer's 23.47, mean GC deviation 2.46 versus 5.91, and average dynamic time warping distance on %MinMax profiles 0.158 versus 0.341. Fitness scores reach Spearman ρ = 0.53 against PaxDb protein abundance in E. coli. On the tufA gene, 63% of low-frequency codon positions match the native sequence, against 28% for CodonTransformer. Fine-tuned for mean ribosome load on the UTR-LM synthetic libraries, it exceeds 5' UTR-LM, RNA-FM, and RNABERT on Spearman, Pearson, RMSE, and MAE.

#Applications

The intended use is heterologous expression: given a protein sequence and a target host, TransCodon generates a coding sequence for synthetic biology, recombinant protein production, and enzyme engineering workflows. Its fitness score doubles as a zero-shot ranker for candidate constructs, and the 5'UTR head supports regulatory sequence design. Weights and training code are released, though neither the GitHub nor the HuggingFace repository carries a license, and the preprint itself is CC BY-NC.

#Impact

TransCodon reframes codon optimization as distribution matching rather than index maximization, and supplies evidence that the reframing matters — most visibly in its recovery of low-frequency codon placement that CAI-driven models erase. The caveats are real: all validation is computational, with no wet-lab expression measurements; a simple background-frequency sampler remains competitive on several distributional metrics; and recovery degraded when training data was pushed to 11 million sequences, suggesting the 12-layer model is near its capacity. As a small, openly downloadable checkpoint spanning 1,436 organisms, it is nonetheless a practical starting point for designing genes for hosts outside the handful that existing tools cover well.