Protein language models pretrained on Rosetta biophysics simulations rather than evolutionary data, then finetuned on small experimental assays.
No providers recorded yet. Browse all providers
METL — Mutational Effect Transfer Learning — is a protein language model framework from the Romero and Gitter labs at the University of Wisconsin–Madison and the Morgridge Institute for Research. It asks what a protein model should learn from before it ever sees experimental data. Models such as ESM-2 pretrain on evolutionary sequence data and excel at recognizing what nature has already sampled. METL pretrains on molecular simulation instead, learning the mapping from sequence to physical energetics that decades of biophysics research have made computable.
The framework runs in three stages. Rosetta models the structures of millions of sequence variants and extracts 55 biophysical attributes from each — molecular surface areas, solvation energies, van der Waals interactions, hydrogen bonding, and related terms. A transformer encoder is pretrained to predict those attributes from sequence alone, forming a representation grounded in energetics rather than phylogeny. That encoder is then finetuned on a small experimental sequence-function dataset — thermostability, catalytic activity, binding, fluorescence — to produce a predictor for that assay. The intended workflow is to pretrain once and finetune cheaply per assay.
Two pretraining scales are provided. METL-Local targets one protein of interest, pretraining on 20 million variants of it with up to five substitutions. METL-Global learns a general representation from 148 diverse base structures with roughly 200,000 variants each. The work appeared as a preprint in March 2024 and was published in Nature Methods in 2025.
METL source models are transformer encoders that take an amino acid sequence and output all 55 Rosetta score terms. METL-Local models hold roughly 2.5 million parameters — 256 embedding size, 3 encoder layers, 4 attention heads — while METL-Global models come in approximately 20-million and 50-million-parameter configurations, the latter requiring about 142 hours on four A100 GPUs. Target models replace the 55-output head with a single functional-score output.
Evaluation spanned 11 sequence-function datasets across eight proteins of varying size and fold — GFP, GB1, DLG4, GRB2, Pab1, PTEN, TEM-1, and Ube4b — against baselines including ESM-2, EVE, Linear-EVE, RaSP, ProteinNPT, and Rosetta's own total score. METL-Local reproduced Rosetta attributes well, reaching a mean Spearman correlation of 0.91 on total score across eight source models, and led on position extrapolation. The paper is candid about where it does not win: evolutionary methods such as Linear-EVE remain competitive or better on many assays, and METL-Global overfits its pretraining set, dropping from 0.85 mean Spearman on in-distribution structures to 0.16 on out-of-distribution ones. Score extrapolation is hard for every method tested. A GB1 data-efficiency analysis found that pretraining on only 1,000 simulated variants and finetuning on 320 experimental points performs comparably to the full-scale version.
METL suits protein engineering campaigns where experimental data is scarce and structure is available: stability engineering, enzyme activity optimization, binder affinity maturation, and fluorescent protein tuning. It fits especially well where evolutionary methods struggle — de novo designed proteins, orphan sequences, or objectives such as thermostability that natural selection never optimized for. Pretrained source and target checkpoints are distributed on Zenodo and load through metl-pretrained, a minimal-dependency PyTorch package, or through the HuggingFace model repository. A HuggingFace Spaces demo and two Colab notebooks, part of the Open Protein Modeling Consortium, cover prediction and user-data finetuning without local setup.
METL establishes molecular simulation as a viable pretraining substrate for protein language models, a different axis from the sequence-scaling that has dominated the field, and the GFP design experiment supplies rare wet-lab validation of a low-N method rather than another retrospective benchmark. Both code repositories are MIT licensed, the Zenodo model archive is versioned and expanding, and the companion metl-sim pipeline makes the simulation stage reproducible. The limits are equally clear: METL-Local requires a per-protein simulation campaign before any modeling begins, METL-Global does not generalize to unseen folds, and a structure of the target protein is a precondition. Its trajectory is tied to that of the simulation methods it depends on — precisely the bet the authors make.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.