Life-Code

Biological sequence models are usually built one molecule at a time: a DNA model, an RNA model, a protein model, each with its own alphabet, tokenizer, and corpus. That partition is convenient for engineering but at odds with the biology, where the three molecules are three views of the same information flowing through the central dogma. Life-Code, from Stan Z. Li's group at Westlake University with collaborators at Zhejiang University, BioMap Research, and the University of Hong Kong, takes the opposite approach and trains a single encoder over a unified representation of all three.

The unification happens in the data pipeline. RNA sequences are reverse-transcribed and amino acid sequences are reverse-translated, so everything arrives at the model as nucleotides. A learned codon tokenizer then merges nucleotide triplets into codon-level tokens, giving the encoder a vocabulary that respects the reading frame of coding regions while still covering non-coding sequence. The result is one model that can be probed with a genomic locus, a transcript, or a protein-coding sequence without changing the interface.

Because a nucleotide-only view discards the structural knowledge that protein language models have accumulated, Life-Code recovers it by distillation: a protein decoder attached to coding regions is trained to match the representations of an off-the-shelf protein language model. This lets folding-related signal reach the shared encoder without ever leaving the nucleotide alphabet.

#Key Features

• Central-dogma data pipeline: Reverse transcription and reverse translation map RNA and protein corpora into nucleotide space, so a single pretraining objective covers all three omics.
• Learned codon tokenizer: A lightweight 8M-parameter module with a convolution-and-unfold operation merges nucleotide triplets into codon embeddings; the learned embeddings recover species-specific codon usage bias and cluster codons that translate to biochemically similar amino acids.
• Hybrid long-sequence encoder: Gated DeltaNet blocks interleaved with occasional self-attention give near-linear scaling over long inputs while retaining the global mixing that pure recurrent stacks lack.
• Structure by distillation: A protein decoder head is supervised against a protein language model teacher, injecting folding signal into the coding-region representations.
• Reverse-complement awareness: Sequence encoding combines forward and reverse-complement features, matching the strand symmetry of double-stranded DNA.

#Technical Details

The encoder follows a BERT-Large-scale layout: 24 layers with embedding dimension 1024 or 1280, reaching up to 500M parameters, built from Gated DeltaNet blocks in an 11:1 ratio with multi-head self-attention blocks and a LLaMA-style macro design (RMSNorm, rotary position embeddings, SwiGLU, FlashAttention). Context length is 4k tokens during pretraining. Training data spans 51.3M DNA sequences from NCBI RefSeq covering roughly 135 species and 35.4B nucleotide bases, 6.5M RNA sequences, and 92M cDNA-amino acid pairs assembled from GenBank and UniRef50; ESM-2 650M serves as the distillation teacher. The tokenizer is pretrained for 100,000 iterations and the encoder for 1M iterations with 15% masking. Reported results include 90.79% average accuracy over the eight Genomic Benchmarks tasks, 73.51% average MCC across the 24 GUE tasks, 95.0 F1 on Spliceator splice-site detection, 94.12% accuracy on ncRNA-protein interaction, and zero-shot protein fitness Spearman correlations of 46.4% on bacterial and 23.8% on human proteins. No public code or weight release accompanies the preprint.

#Applications

Life-Code is aimed at groups that need one representation across a multi-omics analysis rather than a stack of single-molecule encoders: annotating regulatory and coding regions in a genome, scoring splice sites and non-coding RNA interactions, and ranking protein variants by fitness from the underlying coding sequence. The codon-level view is particularly natural for questions that straddle levels — synonymous variant effects, codon optimization, and the coupling between a coding sequence and the protein it produces — where a DNA-only or protein-only model must be told about the other half.

#Impact

Life-Code makes a concrete case that the central dogma can be a modeling principle rather than just a motivation, showing that one encoder trained over a unified nucleotide representation can be competitive with specialist models on DNA, RNA, and protein benchmarks alike. Its combination of codon tokenization, a linear-attention hybrid backbone, and cross-modality distillation offers a template for later multi-omics foundation models. The main practical limitation is availability: at the time of the preprint neither code nor pretrained weights had been released publicly, so independent evaluation of the reported results is not yet possible.