Loading…
Mixed-modality genomic language model encoding protein coding sequences as amino acids and intergenic DNA as nucleotides in native genomic context.
gLM2 is a mixed-modality genomic language model that jointly represents protein and DNA within their native genomic context. Most biological language models specialize in a single modality: protein models such as ESM read isolated amino-acid sequences, while DNA models read raw nucleotides. Both discard the information encoded in how genes are arranged along a genome. gLM2 instead treats a genomic contig as an ordered sequence of elements, tokenizing protein coding sequences (CDS) as amino acids and intergenic sequences (IGS) as nucleotides, with each element prefixed by a strand token. A single model therefore learns protein structure and function signal, regulatory DNA, and the co-evolutionary relationships between neighboring genes.
The model was developed by Tatta Bio with collaborators at the DOE Joint Genome Institute, EMBL-EBI, Seoul National University, and MIT, and introduced in an August 2024 bioRxiv preprint (Cornman, West-Roberts, Camargo, Roux, Beracochea, Mirdita, Ovchinnikov, and Hwang) that was accepted as an ICLR 2025 poster. It was released alongside the Open MetaGenomic (OMG) corpus, the mixed-modality dataset on which it was trained, and is distributed at two scales, 150M and 650M parameters.
By coupling protein and nucleotide modeling on a genomic scaffold, gLM2 occupies a distinct position between protein language models and DNA language models, capturing the multi-gene organization that neither captures alone. This genomic context is the basis for downstream work from the same group, including the FlashPPI interaction predictor, which initializes from gLM2.
gLM2 is a transformer encoder trained with a masked language modeling objective at a 30% masking rate. The 650M-parameter model uses a 4096-token context window and was pretrained on roughly 315B tokens using bfloat16 mixed precision with the AdamW optimizer (learning rate 1e-3, betas 0.9/0.95). Training data comes from the Open MetaGenomic (OMG) corpus, which combines the JGI IMG and EMBL MGnify metagenomic repositories into approximately 3.1 trillion base pairs spanning about 3.3 billion protein coding sequences. Prior to training, the corpus was semantically deduplicated with SemDedup, pruning about 49% of records using an embedding-distance threshold to reduce redundancy and rebalance the data. In the authors' evaluations, gLM2 exceeds ESM2 on most protein tasks and additionally recovers protein-protein interfaces through co-evolutionary signal learned from genomic neighborhoods.
gLM2 targets microbial and metagenomic genomics, where the arrangement of genes carries functional signal that single-modality models miss. Its embeddings support protein and DNA downstream tasks including functional annotation, variant and regulatory analysis, and interaction prediction; the categorical Jacobian provides a route to inferring physical protein-protein interfaces directly from sequence context. Because it produces genomic-context-aware representations, gLM2 serves as a backbone for specialized tools such as FlashPPI for proteome-scale interaction screening, benefiting microbiologists and metagenomics researchers who work with uncharacterized microbial sequences.
gLM2 is the first mixed-modality genomic language model, demonstrating that jointly modeling amino acids and nucleotides on a shared genomic scaffold yields representations competitive with dedicated protein models while adding DNA-level capability and emergent co-evolutionary structure. Paired with the openly released OMG corpus, it has become a reusable foundation for downstream microbial genomics work and was recognized as an ICLR 2025 poster. The model weights, inference code, and training corpus are openly licensed, though the accompanying preprint is released under a non-commercial CC BY-NC 4.0 license and, as an encoder, gLM2 produces representations rather than generating sequences.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.