gLM2

gLM2 is a mixed-modality genomic language model that jointly represents protein and DNA within their native genomic context. Most biological language models specialize in a single modality: protein models such as ESM read isolated amino-acid sequences, while DNA models read raw nucleotides. Both discard the information encoded in how genes are arranged along a genome. gLM2 instead treats a genomic contig as an ordered sequence of elements, tokenizing protein coding sequences (CDS) as amino acids and intergenic sequences (IGS) as nucleotides, with each element prefixed by a strand token. A single model therefore learns protein structure and function signal, regulatory DNA, and the co-evolutionary relationships between neighboring genes.

The model was developed by Tatta Bio with collaborators at the DOE Joint Genome Institute, EMBL-EBI, Seoul National University, and MIT, and introduced in an August 2024 bioRxiv preprint (Cornman, West-Roberts, Camargo, Roux, Beracochea, Mirdita, Ovchinnikov, and Hwang) that was accepted as an ICLR 2025 poster. It was released alongside the Open MetaGenomic (OMG) corpus, the mixed-modality dataset on which it was trained, and is distributed at two scales, 150M and 650M parameters.

By coupling protein and nucleotide modeling on a genomic scaffold, gLM2 occupies a distinct position between protein language models and DNA language models, capturing the multi-gene organization that neither captures alone. This genomic context is the basis for downstream work from the same group, including the FlashPPI interaction predictor, which initializes from gLM2.

#Key Features

• Mixed-modality tokenization: Encodes a contig as interleaved CDS (per-amino-acid tokens) and IGS (per-nucleotide tokens) with strand indicators, letting one model handle both protein and DNA downstream tasks.
• Genomic context and co-evolution: Training on multi-protein genomic neighborhoods lets gLM2 learn cross-protein co-evolutionary signal, enabling prediction of protein-protein interfaces via a categorical Jacobian.
• Competitive protein representations: gLM2 outperforms ESM2 on most protein benchmarks reported by the authors while also supporting DNA-level tasks.
• Two model scales: Available in 150M and 650M parameter versions, with the 650M model using an extended 4096-token context window.
• Fully open weights and code: The pretrained weights and inference code are released on Hugging Face and GitHub under the Apache-2.0 license, and the OMG training corpus is publicly hosted under CC BY-SA 4.0.

#Technical Details

gLM2 is a transformer encoder trained with a masked language modeling objective at a 30% masking rate. The 650M-parameter model uses a 4096-token context window and was pretrained on roughly 315B tokens using bfloat16 mixed precision with the AdamW optimizer (learning rate 1e-3, betas 0.9/0.95). Training data comes from the Open MetaGenomic (OMG) corpus, which combines the JGI IMG and EMBL MGnify metagenomic repositories into approximately 3.1 trillion base pairs spanning about 3.3 billion protein coding sequences. Prior to training, the corpus was semantically deduplicated with SemDedup, pruning about 49% of records using an embedding-distance threshold to reduce redundancy and rebalance the data. In the authors' evaluations, gLM2 exceeds ESM2 on most protein tasks and additionally recovers protein-protein interfaces through co-evolutionary signal learned from genomic neighborhoods.

#Applications

gLM2 targets microbial and metagenomic genomics, where the arrangement of genes carries functional signal that single-modality models miss. Its embeddings support protein and DNA downstream tasks including functional annotation, variant and regulatory analysis, and interaction prediction; the categorical Jacobian provides a route to inferring physical protein-protein interfaces directly from sequence context. Because it produces genomic-context-aware representations, gLM2 serves as a backbone for specialized tools such as FlashPPI for proteome-scale interaction screening, benefiting microbiologists and metagenomics researchers who work with uncharacterized microbial sequences.

#Impact

gLM2 is the first mixed-modality genomic language model, demonstrating that jointly modeling amino acids and nucleotides on a shared genomic scaffold yields representations competitive with dedicated protein models while adding DNA-level capability and emergent co-evolutionary structure. Paired with the openly released OMG corpus, it has become a reusable foundation for downstream microbial genomics work and was recognized as an ICLR 2025 poster. The model weights, inference code, and training corpus are openly licensed, though the accompanying preprint is released under a non-commercial CC BY-NC 4.0 license and, as an encoder, gLM2 produces representations rather than generating sequences.