GeneCLR-DefenseFinder

Two independent signals betray an anti-phage defense gene: the protein itself, since nucleases, ATPases, and SIR2 domains recur across immune systems, and the company it keeps, since defense genes cluster in neighborhoods called defense islands. Aude Bernheim's Molecular Diversity of Microbes lab at Institut Pasteur built a model for each signal in isolation — ESM-DefenseFinder reads amino acids and ignores the neighborhood, ALBERT-DefenseFinder reads the neighborhood and never sees an amino acid — and showed the two score largely non-overlapping populations of protein families.

GeneCLR-DefenseFinder is the model that uses both. Each gene enters as an ESM-2 embedding, and a transformer reads a window of consecutive genes with their orientations and intergenic distances, so a prediction reflects what the protein looks like and what surrounds it. Reported in Science in April 2026, it is the combined approach behind the paper's headline figure of up to 99% precision and 92% recall — and the only one of the three absent from the January 2025 preprint, which described the sequence and context models alone.

The name is the training recipe. CLR is contrastive learning of representations: during pretraining a gene's own embedding is masked out of the context track, and the model is trained so the representation the context produces at that position matches the true protein embedding from a separate focal track. Learning to guess a gene from its neighbors alone forces the context encoder to internalize operon structure before it sees a defense label.

#Key Features

• Two tracks over one representation: A focal track passes each gene's own protein embedding through gated linear units while a context track sees the same embeddings with positions masked, so sequence and neighborhood evidence enter through separate, comparable paths.
• Distance- and strand-aware attention: The context transformer uses a double-stranded attention scheme biased by six pairwise distance features between genes, transformed with asinh, so the model reads real operon geometry rather than a flat list of tokens.
• Runs from a raw genome: Given a nucleotide FASTA, the tool calls genes with Pyrodigal, embeds each protein, tiles the replicon into windows, and returns a per-gene score with no intermediate annotation step.
• Calibrated decision thresholds: Every gene is reported against the best-F1, 1% false discovery rate, and 0.1% false discovery rate cutoffs established in the paper, so users choose a precision target rather than an arbitrary one.

#Technical Details

Proteins are embedded with ESM-2 at 35 million parameters (facebook/esm2_t12_35M_UR50D) and mean-pooled to a 480-dimensional vector per gene. Genomes are tiled into overlapping windows of 64 genes with a stride of 32. The context track is a 12-layer, 8-head transformer with a 1,920-dimensional feed-forward width, matching the 480-dimensional embedding space; both tracks project to 64 dimensions for the contrastive objective, a symmetric InfoNCE loss computed only over masked positions. Fine-tuning swaps in a token classification head that emits a single logit per gene under a binary cross-entropy loss, with LoRA adaptation supported, and per-gene logits are averaged across every window a gene appears in. The released inference checkpoint ships published thresholds of −4.74 for best F1, −0.74 for a 1% false discovery rate, and 8.18 for 0.1%.

Across the study, the sequence, context, and combined approaches reach up to 99% precision and 92% recall on curated benchmarks, with the fusion the strongest of the three. The Science full text is behind a subscription, and the checkpoint is distributed as a single inference file on HuggingFace without an accompanying model card; the DefenseFinder repository documents how to run it.

#Applications

The model ships inside the standard DefenseFinder command-line tool: pip install 'mdmparis-defense-finder[ESMDF]' adds both language model extensions, and -g runs GeneCLR alongside the curated HMM search while -G runs it alone. Because gene calling is internal, it takes a nucleotide genome FASTA rather than a protein file — the practical trade against ESM-DefenseFinder, which scores a bare protein sequence. The typical use is triage: rank the hypothetical proteins in an accessory genome, take those clearing a stringent false discovery threshold, and clone them for phage challenge. DefensePredictor targets the same workflow with a different fusion of embeddings and neighborhood features.

#Impact

Applied to more than 32,000 bacterial genomes, the models estimate that roughly 1.5% of a typical bacterial genome is devoted to defense, and that over 85% of the protein families they predict as defensive remain uncharacterized. The study validated 12 anti-phage systems experimentally in Escherichia and Streptomyces, and publishes an interactive catalog of more than 19,000 candidate operon families. That catalog, rather than any single benchmark number, is the deliverable: a ranked, browsable queue of neighborhoods a wet lab can order and test. The constraints are practical — assembled genomes are required, no protein-FASTA path exists, and the released checkpoint carries no separate documentation of its training data.