Nammu

Metagenomic sequencing has expanded more than 500-fold since its broad adoption in 2011, yet over half of the resulting sequences still lack a confident functional assignment. Much of the signal that could resolve them is contextual: synteny, operon structure, and the regulatory landscape of the intergenic DNA a protein sits in. Assembled contigs preserve that context across tens of thousands of base pairs, but attention scales quadratically with sequence length, so transformer-based models such as gLM2 operate over a fixed 4,096-token window and must chop long contigs into chunks that are embedded separately and averaged back together.

Nammu, from Oregon State University and UC Santa Barbara, replaces attention with a state space model to remove that ceiling. It is a 167M-parameter bidirectional Mamba encoder trained on the OpenMetaGenomic (OMG) corpus with a 20,000-token context window — long enough to hold more than 95% of OMG contigs whole, spanning multiple coding and non-coding regions in a single representation. Because Mamba scales linearly in sequence length, that context comes without the quadratic cost that constrains transformer encoders.

The model was posted as a bioRxiv preprint in July 2026 under a CC BY license. Its authors describe it as the first state space–based genomic language model built for long-context mixed nucleotide–protein representation learning: earlier SSM work in biology, including Caduceus, ProtMamba, and LC-PLM, remained either DNA-only or protein-only, while gLM2 was the only prior mixed-modality genomic language model and is a transformer.

#Key Features

• Long-context modeling without windowing: A 20K-token context lets an entire metagenomic contig — several genes plus the intergenic DNA between them — be encoded in one forward pass, rather than reconstructed by averaging sliding-window chunks.
• Mixed-modality tokenization: Sequences are tokenized at base-pair resolution, with coding regions as amino acids, intergenic regions as nucleotides, and explicit strand-orientation tokens, following the OMG encoding scheme also used by gLM2.
• Bidirectional BiMamba architecture: Eighteen blocks process each input forward and reversed along the sequence dimension and sum the results, sharing input and output projections across directions while keeping SSM parameters independent. No reverse-complement symmetry is assumed, since interleaved amino acid and nucleotide segments do not obey it.
• Two-stage curriculum pretraining: The model is first trained as a protein language model, then further pretrained on mixed-modality contigs, so protein-level representations are learned before genomic context is introduced.
• Token efficiency: Nammu matches or exceeds gLM2 on most benchmarks while training on 230B tokens against gLM2's 315B, over the same corpus.

#Technical Details

Both pretraining stages use masked language modeling. Stage one covers 64M protein sequences drawn from OMG_prot — the 207M-sequence protein subset clustered at 50% identity — with 15% masking for 500K steps under AdamW, batch size 128, and a linear learning rate decay. Stage two runs 250K steps over 32M mixed-modality contigs with 30% masking and a cosine schedule from 1e-3 to 1e-4. Training used a single NVIDIA GH200 system over roughly two months.

On the Diverse Genomic Embedding Benchmark (DGEB), Nammu beats gLM2 on 8 of 14 protein tasks and 6 of 8 DNA tasks, while dedicated unimodal models retain an edge on their own modality — ESM-2 is consistently stronger on protein tasks and Nucleotide Transformer v2 leads on several DNA tasks. The contextual gains appear on contig-level evaluations. On a CAMI II marine sample (102K contigs, 246 species), zero-shot transfer of Nammu embeddings through a ridge classifier reaches 0.46 MCC at class level versus 0.25 for gLM2, and 0.09 versus 0.03 at species level; restricted to contigs longer than 4,096 tokens the gap widens to 0.95 versus 0.80 at class level. For KEGG Orthology prediction across 67,172 annotated ORFs from more than 3,000 deep-sea methane seep MAGs, Nammu reaches 0.30 MCC on context-free protein embeddings against 0.25 for gLM2 and 0.18 for ESM-2.

#Applications

Nammu targets the annotation bottleneck in environmental and host-associated microbiome studies, where assemblies are fragmented and reference-similarity methods leave most sequences unassigned. Microbial ecologists can use its contig embeddings for taxonomic classification and KEGG Orthology assignment through lightweight downstream classifiers, without fine-tuning the encoder. Because the model reads whole contigs rather than isolated open reading frames, it is best suited to questions where gene neighborhood carries the signal — characterizing uncultured lineages, biosynthetic gene clusters, and functionally uncharacterized microbial "dark matter."

#Impact

Nammu is a demonstration that architecture choice, not just scale, governs how much biological context a genomic model can exploit: it outperforms a comparably sized transformer trained on the same corpus while consuming 27% fewer tokens, and its margin grows precisely where context is longest. That result strengthens the case for linear-scaling sequence models in metagenomics and for the OMG corpus as a general pretraining resource. Practical caveats remain. The work is an unreviewed preprint; the authors identify scaling in model size and context, evaluation below 40% sequence identity, and application to metagenomic bin refinement as open directions. Only training and benchmarking code has been released, under an MIT license through an anonymized review repository, and no pretrained weights have been published, so the model cannot yet be used off the shelf.