LassoESM

A lasso peptide is a 15–20 residue chain that its cyclase ties into a threaded [1]rotaxane knot, and whether a given variant gets tied at all is not something the sequence obviously announces. The fusilassin cyclase FusC will accept one substitution at position 14 and reject its neighbor; the rules behind that are unwritten, and the labeled data that might reveal them run to hundreds of assayed variants, not millions. A general-purpose protein language model is little help here — it learned its statistics from full-length globular proteins, where an 18-residue knotted macrolactam is a rounding error.

LassoESM answers this by domain adaptation rather than a new architecture. The authors took ESM-2 at 650 million parameters and continued masked-language-model pretraining on 4,485 unique high-scoring lasso core peptides mined from GenBank by RODEO, updating all weights so the model's residue statistics move from proteins in general to this one peptide class. The resulting checkpoint is then used frozen: mean-pooled embeddings feed small task heads — an SVM, an MLP regressor, a cross-attention pairing network — rather than the language model being re-fit for each new question.

The model comes from Xuenan Mi, Susanna Barrett, Douglas Mitchell and Diwakar Shukla at the University of Illinois Urbana-Champaign, with Mitchell also at Vanderbilt University. It was posted to bioRxiv in October 2024 and published in Nature Communications in September 2025.

#Key Features

• Domain-adaptive pretraining on a small corpus: Full-parameter continued pretraining on 4,485 lasso core sequences, with GaLore gradient low-rank projection keeping the run inside a single 12 GB RTX 3090.
• Frozen embeddings across unrelated tasks: One checkpoint supplies representations for substrate tolerance classification, cyclase–peptide pairing, and activity regression; only the small downstream heads are fit per task.
• Cross-attention cyclase–peptide pairing: Cyclase embeddings from generic ESM-2 and peptide embeddings from LassoESM are combined through a cross-attention layer, which weights the cyclase residues likely to contact the substrate instead of concatenating two independent vectors.
• Useful in the data-scarce regime: Trained on 20% of the fusilassin variant set — 224 sequences — the SVM head still reaches 0.78 AUROC and 0.71 balanced accuracy.
• Wet-lab closed loop: Predictions were tested by cell-free biosynthesis and MALDI-TOF-MS, and the results fed back into retraining.

#Technical Details

The architecture is ESM-2's 33-layer encoder at 650M parameters, retrained with 15% token masking for 20 epochs at a 5 × 10⁻⁵ learning rate and batch size 8; the final-layer 1,280-dimensional per-residue vectors are averaged into a sequence embedding. On 1,121 fusilassin variants and 552 microcin J25 variants labeled as substrate or non-substrate, an SVM over LassoESM embeddings outperformed the same head over generic ESM-2 embeddings and one-hot encoding for both cyclases, which share only 22% sequence identity. The pairing model was trained on 6,599 RODEO-derived cyclase–peptide pairs against an equal number of acceptor-residue-mismatched negatives, reaching 0.887 accuracy on a random split and 0.828 when both cyclase and peptide were held out entirely. For RNA polymerase inhibition, an MLP regressor over LassoESM embeddings of 8,885 ubonodin variants reached Pearson 0.83 and Spearman 0.78 with a mean absolute error of 1.31, against 2.20 for the task-specific DeepLasso CNN-LSTM; a matching AdaBoost regressor over a klebsidin variant set showed the same ordering.

#Applications

The immediate use is triage in lasso peptide engineering: scoring which members of a combinatorial core-peptide library a particular cyclase will process before committing them to cell-free biosynthesis, choosing a cyclase capable of producing a designed peptide, and ranking variants for RNA polymerase inhibition when the goal is an antibacterial scaffold. Because the embeddings are frozen and mean-pooled, a downstream classifier is a few lines of scikit-learn over a matrix of vectors, which puts it within reach of natural-product chemists and synthetic biologists without deep learning infrastructure.

#Impact

The experimental arm is what distinguishes this from an embeddings benchmark: 196 fusilassin variants were synthesized and assayed across three human-in-the-loop rounds, with balanced accuracy on freshly proposed library members rising from 66% to 75% to 84% as each round's results were folded back into training. The limits are equally concrete. The pretraining corpus is small enough that coverage of underexplored lasso families is uncertain, the model scores sequences without explaining the structural basis for cyclase selectivity, and every application requires its own labeled set to fit a head. The weights are on Hugging Face and the code on GitHub, but neither carries a license file; only the Zenodo archive of the code is explicitly licensed, under CC BY 4.0.