Bacformer

Bacteria encode their adaptations in how genes are arranged, not only in the genes themselves: functionally related genes sit contiguously in operons, transcription factors mostly regulate proximal genes, and distance from the origin of replication carries information about gene identity. Sequence models that embed proteins or DNA windows in isolation discard that organisation. Bacformer instead represents an entire bacterial genome as an ordered sequence of proteins and learns the compositional and positional grammar that governs how those proteins are arranged across the bacterial kingdom.

Developed by researchers at the University of Cambridge Centre for AI in Medicine and the Victor Phillip Dahdaleh Heart & Lung Research Institute, with collaborators at EPFL and the SIB Swiss Institute of Bioinformatics, Bacformer embeds each protein in a genome with a pretrained protein language model (ESM-2), treats that embedding as a single token, and passes the whole proteome through a transformer in one forward pass. The output is a set of contextualised protein embeddings, each conditioned on every other protein present in the organism.

The model was posted as a bioRxiv preprint in July 2025 and has not yet been peer reviewed. Code, tutorials, and pretrained checkpoints are released under Apache 2.0.

#Key Features

• Genome-scale context window: A single forward pass covers up to 6,000 proteins, enough to hold more than 98% of the training corpus genomes in full context, with learnable contig embeddings distinguishing chromosome from plasmid.
• Protein-family vocabulary: Rather than a fixed token dictionary, proteins are clustered in embedding space into 50,000 families used as pretraining labels — an approach that accommodates open-ended protein diversity and recovers established orthology, scoring 74% and 88% normalised mutual information against eggNOG and MMseqs2 respectively.
• Experimentally validated operon prediction: Zero-shot operon calls from pairwise embedding similarity reached AUROC 0.76–0.92 against operons the authors annotated with new long-read RNA sequencing of five species, including Mycobacterium tuberculosis and Pseudomonas aeruginosa.
• Phenotype inference with gene attribution: Genome-level embeddings support classifiers for 139 phenotypic traits, and gradient-based attribution ranks the genes driving each call, recovering canonical sporulation and niche-adaptation genes alongside uncharacterised candidates.
• Conditional genome generation: An autoregressive variant generates full protein-family sequences from a short prompt, and a prepended property token conditions generation on traits such as oxygen requirement or optimal growth temperature.

#Technical Details

Pretraining used a curated corpus of 1.3 million metagenome-assembled genomes spanning roughly 25,000 species, 1,500 orders, and 70 biomes — about 3 billion proteins and 225 million contigs drawn from MGnify, SPIRE, and NCBI RefSeq — followed by continued training on 48,932 complete RefSeq genomes. The base model is a 12-layer transformer with a hidden dimension of 480 and roughly 27M parameters, using rotary positional embeddings over genomic order; a 300M-parameter Bacformer Large checkpoint was released subsequently. The masked objective hides 15% of proteins and predicts their family from context; causal checkpoints are also released.

On benchmarks, fine-tuned Bacformer reached a median AUROC of 0.81 for gene essentiality across held-out genomes from the DEG database, outperforming fine-tuned ESM-2 and the DNA language model Evo. It beat non-contextual ESM-2 embeddings on zero-shot protein-protein interaction recovery against STRING labels across 2,088 held-out genomes, on protein function prediction over 503,295 held-out proteins across 188 KEGG classes (average silhouette width 0.71 versus −0.05), and on strain clustering by 5–13% ARI. Novel interactions predicted in P. aeruginosa PAO1 were checked with AlphaFold 3, which assigned them significantly higher ipTM scores than random protein pairs.

#Applications

Bacformer suits microbial genomics groups working past the limits of single-gene annotation: annotating operon structure and gene essentiality in poorly characterised isolates, prioritising drug targets in pathogens, mapping functional networks without transcriptomic data, and inferring phenotypes such as motility, growth temperature, or carbon-source utilisation directly from a draft assembly. Because genome embeddings live in a shared space, an unannotated MAG can be matched by nearest-neighbour search against 1.3 million embedded genomes with associated metadata. The generative mode extends to synthetic biology, completing genomes from partial metagenomic protein sets and sketching trait-conditioned genome designs.

#Impact

Bacformer stakes out a representation level that sits between protein language models and DNA language models: the proteome as an ordered document. That framing lets a small model exploit genomic context that per-protein embeddings cannot see, and the accompanying BacBench suite gives the field a common yardstick for bacterial genomics models across essentiality, operon, interaction, resistance, and trait prediction. The main caveats are the ones the work states plainly: it is a preprint, generative outputs are protein-family sequences rather than validated synthetic genomes, and the phenotype annotations released for the full corpus are model predictions rather than measurements.