Genomic language model reading bacterial gene neighborhoods as sentences of protein-family tokens to predict anti-phage defense function.
No providers recorded yet. Browse all providers
Anti-phage defense genes cluster in bacterial genomes, in neighborhoods called defense islands. That observation has driven most defense system discovery for a decade, usually through a family-wise "defense score" counting how often a protein family is found near a known defense gene. It is a blunt instrument: it treats the neighborhood as a bag of genes and cannot learn what a defensive context looks like.
ALBERT-DefenseFinder, from Aude Bernheim's lab at Institut Pasteur, replaces that count with a genomic language model. Each gene is mapped to a token — its family at 50% amino acid identity — and a stretch of 64 consecutive genes becomes a sentence. The model is pretrained by masked language modeling on those sentences, then fine-tuned to label each token as defensive or not. The premise, following the word-to-gene-family analogy Microbial Gene NLP established with word2vec embeddings, is that a transformer can learn syntax in gene order that a co-occurrence statistic cannot.
Crucially, the model never reads an amino acid. That makes it blind to sequence similarity and therefore able to nominate defense proteins whose folds are unlike anything in the training set — the exact failure mode of its sibling ESM-DefenseFinder. Six novel anti-phage systems were validated in the lab on that basis.
The corpus is 10,796 diverse actinobacterial genomes, including 2,880 Streptomyces, whose 54 million proteins were clustered at 50% amino acid identity into roughly 4.2 million families. Pretraining used masked language modeling over 6.7 million fragments of up to 64 genes — 432 million tokens — with 95% of genomes for training. The masked-language head was then swapped for a token classification head and fine-tuned with LoRA at rank 64 and alpha 128 on 20,000 fragments, half sampled to upweight rare defense system types, under a class-frequency-weighted cross-entropy loss. Genes from validation or test system types were masked out of the training labels. Per-gene scores are averaged across the fragments a gene appears in, then summarized to family level at the 90th percentile.
On the held-out Actinomycetota test set of 230 positive and 36,466 negative families, ALBERT-DefenseFinder reached 88.4% AUROC and 7.7% average precision, ahead of the word2vec-style context baseline at 85.9% and 4.0% and the conventional defense score at 80.4% and 4.4%. Its architectural limit is vocabulary scale: a full-RefSeq instance would need more than 1.5 million tokens, so the model was confined to a single phylum. A third sibling, GeneCLR-DefenseFinder, fuses genomic context with protein sequence and reaches up to 99% precision and 92% recall on curated benchmarks in the published version.
The workflow is designed to hand a wet lab a synthesis order. From the model's predictions the authors auto-generated over 200 candidate multi-gene systems, excluded anything resembling a mobile genetic element, and picked ten with diverse predicted biochemistry. Two distinct homologs of each were cloned under a constitutive promoter into Streptomyces albus and challenged with twelve phages. Two candidates proved toxic; of the remaining eight, six reduced plaque-forming units more than a hundredfold against at least one phage — the systems named Ceres, Geb, Veles, Prithvi, Ukko, and Oshun, with Ukko conferring roughly thousandfold protection.
The mechanistic novelty of those six systems is the point. Most of their Pfam domains had never been described in anti-phage defense: Prithvi is annotated as a FAD/NAD-dependent halogenase, Ukko comprises four enzymes suggesting a tryptophan-derived biosynthetic pathway, and Geb is a single 150-residue protein with no annotated domain at all. Within Actinomycetota, the model accounts for 72% of the 12,114 novel candidate defense families the study predicts, a fourfold expansion over the 3,171 families DefenseFinder recognizes in the phylum. Across more than 32,000 genomes the study validated 12 anti-phage systems in Escherichia and Streptomyces and published a catalog of over 19,000 candidate operon families. The trade-off against its siblings is stark: this model needs an assembled genomic neighborhood and a phylum-scoped vocabulary, where a sequence model needs only a FASTA file. Its weights and inference code have not been released — the accompanying repository holds supplementary tables — and it is ESM-DefenseFinder and GeneCLR-DefenseFinder that ship in the DefenseFinder toolchain.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.