Metalic

The first round of a protein engineering campaign has to pick variants for a protein nobody has measured yet. The usual answer is to rank mutations by how likely a protein language model thinks the mutated sequence is, which quietly assumes that sequence likelihood tracks function — an assumption that holds on average across a benchmark and can fail badly on the one protein a lab actually cares about. Metalic, from InstaDeep, replaces that assumption with data borrowed from other proteins: it treats the deep mutational scans already published in ProteinGym as a distribution of tasks and meta-trains a single predictor across them, learning once how language-model embeddings relate to measured fitness and reusing that relationship on proteins and assays it has never seen.

Metalic is an in-context predictor rather than a per-assay regressor. At inference it receives a support set of labelled variants — possibly empty — together with a query set of unlabelled candidates, and ranks the query set in one forward pass; the task arrives as context instead of being written into weights. Because meta-training has already taught it how to use that context, the zero-shot case works: with no support set at all, the fixed checkpoint reaches 0.484 Spearman on eight held-out ProteinGym single-mutant tasks against 0.121 for the ESM-2 8M backbone it embeds with. Removing the meta-training stage collapses that correlation to roughly zero, which places the transfer in meta-training rather than in the backbone or the architecture.

The method appeared as a preprint in October 2024 and was published at ICLR 2025.

#Key Features

• Meta-learning over assays: Training optimizes a cross-task objective over 113 single-mutant and 68 multi-mutant ProteinGym landscapes, so held-out proteins and held-out assays are the evaluation, not held-out folds of the same experiment.
• Zero-shot without a likelihood assumption: With an empty support set the model still conditions on the unlabelled query sequences, an unsupervised form of in-context adaptation that needs no fitness labels for the target protein.
• Fine-tuning bolted on after meta-training: When support labels exist, 100 gradient steps on subsampled support/query splits adapt the checkpoint further — and because fine-tuning is deliberately excluded from the meta-training objective, no meta-gradients are ever computed.
• A small backbone by design: Per-residue embeddings come from the third layer of ESM2-8M, giving roughly 36M parameters in total and matching models around 18 times larger in the low-data regime.
• Metalic-AuxIF variant: Also written Metalic-IF1, this configuration additionally embeds zero-shot predictions from ESM-IF1 as an auxiliary input, raising totals to about 170M parameters and improving results at every support size.

#Technical Details

The predictor is a ProteinNPT-style axial-attention stack: five layers, embedding dimension 768, four attention heads, and a four-layer MLP head. Support-set fitness values are projected into the residue embedding dimension by a linear layer while query-set fitness slots share a single learned embedding, and the two are concatenated along the sequence dimension. Axial attention alternates within-sequence and across-sequence attention, cutting self-attention cost from O(K²L²) to O(K²+L²) for K sequences of length L. Training uses a preference-based loss — pairwise binary classification of which of two query sequences is fitter — for 50,000 updates on a single A100-80GB, two to eight days per seed, with contexts of at most 171,000 tokens and proteins capped at 750 residues. Evaluation uses support sizes of 0, 16 and 128 with a query set of 100. On held-out single-mutant tasks Metalic scores 0.482 ± 0.002 zero-shot, ahead of VESPA at 0.464 and ESM-1v at 0.437, then 0.484 at 16 shots and 0.552 at 128; Metalic-AuxIF reaches 0.498, 0.500 and 0.556.

#Applications

The intended user is a protein engineer choosing which variants to build when the assay is new and the label budget is small — early rounds of enzyme thermostability work, binding affinity optimization, or any campaign where a deep mutational scan has not yet been run. A single checkpoint serves arbitrary new targets, so a group can prioritize a library without training anything, then fine-tune on the first 16 or 128 measurements as they come back. Inference fits comfortably on one GPU.

#Impact

Metalic makes the case that labelled fitness data from unrelated proteins is a more useful resource than a larger unlabelled pretraining corpus when the target assay is data-poor, and it does so cheaply enough that the comparison is hard to dismiss. The limits are stated plainly by the authors: the state-of-the-art claims cover zero-shot and low-data single-mutant ProteinGym, and on multi-mutant tasks Metalic is competitive but not best, scoring 0.436 zero-shot where ESM-IF1 reaches 0.590 — a gap they attribute to the small number of multi-mutant meta-training landscapes, since performance rises as those tasks are added. Released checkpoints cover the zero-shot single-mutant setting only, and evaluation is in silico on benchmark assays rather than prospective wet-lab validation.