Loading…
Bacterial proteome foundation model that learns contextualized gene and whole-genome representations from tens of thousands of complete genomes.
No providers recorded yet. Browse all providers
Most protein language models reason about one protein at a time, learning from amino acid sequences in isolation. But in bacteria, gene function is shaped by genomic context: operon structure, neighboring genes, and the broader organization of the proteome all carry functional signal that per-protein models discard. BacPT (Bacterial Proteome Transformer) is a foundation model designed to capture this context by modeling an entire bacterial proteome as an ordered sequence of proteins rather than as independent entities.
Developed by researchers at the University of Florida, the model was first disclosed in a March 2025 bioRxiv preprint on learning gene interactions from entire bacterial proteomes and expanded in a March 2026 preprint that gave it the BacPT name and broadened its evaluation. It is trained in a self-supervised fashion on tens of thousands of complete bacterial genomes spanning diverse taxa. Instead of predicting masked amino acids, it learns to reconstruct corrupted per-protein ESM2 embeddings from their genomic context, producing contextualized gene embeddings and functionally rich whole-genome representations without task-specific labels.
The result is a model that sits a level above conventional protein language models, capturing local gene-neighborhood signal and genome-wide organization that improve a range of downstream functional predictions across enzymology, secondary metabolism, metabolism, and microbial ecology.
Each gene is first encoded with ESM2 protein embeddings; a transformer then operates over these per-protein vectors to integrate local and genome-wide context, reconstructing deliberately corrupted embeddings from their surroundings. Training data were curated by clustering protein-family representations across 33,140 bacterial genomes, yielding a final pretraining set of 28,133 complete genomes selected for taxonomic and functional diversity. Two variants are implemented: BacPT-small, a RoBERTa backbone with relative key-query position embeddings — hidden size 480, 10 layers, and 5 attention heads — trained end-to-end over whole genomes of up to 5,000 genes; and BacPT-large, a RoFormer backbone with rotary position embeddings trained in two stages, short-contig pretraining followed by whole-genome fine-tuning. The learned representations are evaluated on downstream tasks spanning enzyme activity, BGC detection, metabolic trait prediction, and ecological interaction outcomes.
BacPT is suited to microbiologists, genome annotators, and computational biologists working at the scale of complete bacterial genomes. Its contextualized embeddings can power functional annotation of poorly characterized genes, prioritization of biosynthetic gene clusters for natural-product discovery, prediction of metabolic capabilities, and modeling of interactions within microbial communities. Because representations are produced unsupervised, the model can be applied as a feature extractor across many downstream tasks without retraining from scratch.
By extending the foundation-model paradigm from individual proteins to entire proteomes, BacPT targets functional signal that per-protein language models structurally cannot access. Its emphasis on genome-level context positions it within an emerging line of work on whole-genome bacterial language models, offering a reusable representation layer for microbial genomics. Pretraining code and the underlying model and dataset classes are published under an MIT license, though pretrained checkpoints are not distributed with the repository. As a preprint, its benchmark advantages await peer review and broader independent evaluation.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.