BGC-Transformer

Biosynthetic gene clusters (BGCs) are contiguous stretches of genome in which functionally related genes cooperate to build a complex natural product — antibiotics, antifungals, immunosuppressants, anticancer agents. Finding them in genome sequence is the entry point to natural-product discovery, and the established tools do it with curated rules: profile HMMs for known biosynthetic domains, plus hand-built logic for how those domains combine into recognizable cluster types. That works well for cluster classes that have been seen before and poorly for anything else.

This framework, released as bgc-transformer by Myco Umemura's group at the Kyoto Institute of Technology with Tomohisa Kuzuyama's group at the University of Tokyo, takes a linguistic view instead. Each Pfam functional domain is treated as a word and each genome or cluster as a sentence, and a RoBERTa masked language model is trained to predict which domain belongs in a masked position from its surrounding domain context. Nothing about biosynthetic chemistry is encoded by hand; the grammar of domain arrangement is learned from how domains actually co-occur along genomes.

The design consequence is the interesting one. Because the model is generative over domain vocabulary, it can propose domains that should fit a cluster context but are absent from the cluster as annotated. The authors tested this on the diterpenoid cyclooctatin: a model trained on whole genomes predicted domains that neither the original BGC nor a BGC-trained model contained, and heterologously expressing one of those predicted domains in Streptomyces albus alongside the cyclooctatin biosynthetic genes produced a previously unknown cyclooctatin derivative.

#Key Features

• Domains as tokens: A 19,523-token vocabulary of unique Pfam domains turns genome annotation into text, so a standard transformer language model can learn positional and contextual relationships among biosynthetic functions.
• Four training scales: Separate models are pretrained on progressively broader corpora — curated bacterial BGCs, Actinomycetes genomes, all bacterial genomes, and bacterial plus fungal genomes — which lets the effect of training breadth on prediction and design be measured directly.
• Masked-domain prediction: The core inference task ranks candidate domains for a masked position, evaluated against experimentally validated clusters rather than only against held-out sequence.
• Generative cluster design: The same masked-prediction machinery proposes domains absent from a known cluster, producing candidate artificial BGCs — one of which was validated in vivo.
• Leakage-safe evaluation: Training and evaluation splits are formed by MinHash-Jaccard grouping of similar domain sequences, preventing near-duplicate clusters from appearing on both sides of the split.

#Technical Details

The models use a RoBERTa architecture with 8 hidden layers, 16 attention heads, a hidden dimension of 1,024, a maximum input of 512 tokens, and 105,781,515 trainable parameters. The four corpora are Dataset I, 8.6M tokens from 239,021 bacterial BGCs predicted in the antiSMASH database; Dataset II, 13.8M tokens from 2,664 Actinomycetes strains; Dataset III, 48.9M tokens from 9,748 bacterial strains annotated in RefSeq; and Dataset IV, 70.5M tokens from 11,884 bacterial and fungal strains. Evaluation on 2,492 experimentally validated BGCs from MIBiG placed more than 50% of true domains first and more than 75% within the top 10 candidates, and compound-class classification exceeded 70% accuracy for major classes including polyketides and terpenes.

#Applications

The immediate use is genome mining: scoring and completing BGC annotations in newly sequenced bacteria and fungi, including taxa whose cluster architectures are underrepresented in rule-based databases. The generative mode supports rational design of artificial BGCs — proposing accessory domains, such as tailoring enzymes, whose addition to a known pathway is likely to yield a modified product. That makes it a hypothesis generator for combinatorial biosynthesis and synthetic biology groups engineering new natural-product analogues in heterologous hosts.

#Impact

The cyclooctatin experiment is what separates this from purely in-silico BGC prediction: a model-proposed domain was expressed and produced a new compound, which is direct evidence that a domain-level language model captures something chemically actionable rather than merely statistical. Two limitations bound the work. The code and trained models are released under CC BY-NC 4.0, so commercial use is not permitted, and the models see only domain identity and order — transcriptional, structural, and phylogenetic signal that constrains real cluster function is absent, an extension the authors identify as future work.