cdsFM

Leucine is written six different ways in the genetic code. A protein language model sees the same leucine six times over and cannot tell them apart; a nucleotide language model sees eighteen bases and carries no marker of where one codon ends and the next begins. Yet the choice among synonymous codons is not arbitrary — it shapes mRNA stability, translation speed, co-translational folding, and ultimately how much protein a cell makes. cdsFM closes that gap by making the codon itself the token: a vocabulary of 64 codons plus five special symbols, so a synonymous substitution is a single-token edit the model is trained to reason about.

cdsFM is a family, and this entry covers both of its members, since they share a corpus, a tokenizer, a codebase and a paper. EnCodon is a bidirectional encoder trained with masked language modeling, used for scoring and for embedding sequences; DeCodon is an autoregressive decoder trained with causal language modeling, conditioned on an organism token prepended to every sequence so that generation can be steered toward a target species. The suite was developed at the Arc Institute with UCSF, UC Berkeley and McGill University, and posted to bioRxiv in October 2024.

The name invites confusion with CodonFM, and the two are genuinely distinct. CodonFM is NVIDIA's 2025 release, trained with Arc on a much larger RefSeq corpus and shipping Encodon checkpoints only; cdsFM is the earlier lineage from Hani Goodarzi's lab, with its own pretraining run and its own DeCodon models. Shared author names reflect a real lineage, not shared weights.

#Key Features

• Codon-resolution tokenization: A 64-codon vocabulary keeps reading frame implicit and synonymous variation explicit, which neither amino-acid nor single-nucleotide tokenization can do.
• Encoder and decoder in one suite: EnCodon supplies pseudolikelihoods and embeddings for scoring tasks; DeCodon supplies conditional generation, making the pair usable for both variant interpretation and sequence design.
• Organism-conditioned generation: DeCodon takes an NCBI taxonomy ID as its first token, so coding sequences can be generated in the codon dialect of a chosen host.
• Two-stage eukaryotic adaptation: Because the pretraining corpus is 98.7% bacterial, every checkpoint has a second-stage variant continued on eukaryotic sequences, denoted "Ada".
• A purpose-built expression dataset: The authors measured overall abundance and cell-surface expression across synonymous variants of SLC22A1, GPR68 and KCNJ2, giving the fine-tuning task a matched experimental readout rather than a proxy.

#Technical Details

Both architectures are rotary-position transformers with a maximum context of 2,048 codons (6,144 nucleotides), covering over 99.5% of coding sequences. Pretraining used 60 million coding sequences from more than 5,000 species aggregated from NCBI Genomes, of which 59.4 million (98.7%) are bacterial, for more than 1,000,000 optimization steps. Three EnCodon scales were trained — 80M, 620M and 1B parameters — alongside DeCodon at 200M and 1B. On zero-shot pathogenicity ranking over 48,283 ClinVar single-nucleotide variants, the eukaryotic-adapted models outperformed CaLM, CodonBERT, HyenaDNA, Nucleotide Transformer and DNABERT-2. Clustering embeddings of 7,264 human Ribo-seq ORFs, DeCodon (200M) reached a mean KNN purity of 24.57% against 21.45% for the 2.5B multispecies Nucleotide Transformer. With LoRA fine-tuning, DeCodon (1B)Ada improved mRNA stability prediction by 5% over that same baseline. The released checkpoints are EnCodon at 80M and 620M and DeCodon at 200M, each with its eukaryotic-adapted counterpart; the 1B models reported in the paper are not among them.

#Applications

The encoder side serves variant interpretation: EnCodon fine-tuned on the synonymous-variant expression data generalized to 513 held-out SLC22A1 variants, and applied to ClinVar synonymous variants of uncertain significance it nominated candidates that the authors then confirmed in a cell-based assay. The decoder side serves design — codon optimization for recombinant protein production and mRNA therapeutics, where the host organism is known and the objective is a coding sequence that expresses well in it. Both are directly usable through the AutoEnCodon and AutoDeCodon wrappers in the released package.

#Impact

cdsFM is one of the clearest arguments that coding sequence deserves its own foundation-model resolution rather than being handled as protein or as generic DNA, and it is the direct antecedent of the larger NVIDIA–Arc CodonFM release that followed a year later. Its limits are worth stating plainly: the models see coding sequences only, so untranslated regions and their regulatory content are outside their scope; the heavily bacterial corpus makes the adaptation stage a requirement rather than a refinement for eukaryotic work; the largest checkpoints described in the paper have not been released; and the work remains a preprint that has not been through peer review.