ESM-IF1

As of January 2022 the Protein Data Bank held structures for fewer than 53,000 unique sequences clustered at 50% identity, against more than 50 million UniRef clusters at the same threshold. That shortfall — not the architecture — was the binding constraint on inverse folding, the task of writing a sequence that will fold into a given backbone.

ESM-IF1 answers with back-translation, the trick machine translation reaches for when parallel text is scarce: manufacture the missing half of each pair with a model running the other direction. Meta AI's protein team, with collaborators at UC Berkeley and NYU, predicted structures for 12 million UniRef50 sequences using AlphaFold 2 and trained on those alongside the experimental set — roughly 750 times more structures than experiment alone supplies. Hsu et al. presented the result at ICML 2022 and released the model that April.

The paper names the architecture GVP-Transformer: geometric vector perceptron layers extract rotation-invariant features from the backbone atom graph, and an autoregressive encoder-decoder transformer reads them and emits residues one position at a time. "ESM-IF1" (sometimes ESM-IF) is the released checkpoint's name; the paper frames the work as inverse folding, not as an entry in the protein language model line that produced ESM-1b and ESM-1v. The extra data is what makes the hybrid worth building: the 1M-parameter pure-GNN baseline gets worse on the predicted structures, while larger models with transformer layers keep improving.

#Key Features

• Sequence design from backbone coordinates: From N, Cα and C positions alone the model samples complete sequences for a fixed backbone, recovering 51.6% of native residues on structurally held-out CATH chains and 72% at buried core positions.
• Tolerance for missing coordinates: Span masking during training — contiguous spans up to 30 residues, up to 15% of input coordinates — lets it design into gaps where a backbone is unresolved or deliberately left open.
• Generalization beyond single chains: Trained only on single chains, it still uses complex context — conditioning on a full complex backbone drops perplexity on the target chain from 6.32 to 3.81, and two conformations lower perplexity at locally flexible residues.
• Zero-shot mutation scoring: The log-likelihood ratio between mutant and wild-type sequence, given an experimental structure, predicts complex stability, binding affinity and insertion effects with no task training.

#Technical Details

The released checkpoint has 142M parameters across 20 layers — 4 GVP-GNN encoder, 8 transformer encoder and 8 transformer decoder — and accepts up to 512 residues. The predicted training set was built by ranking UniRef50 sequences on MSA Transformer distogram LDDT scores, then folding the top 12 million of 500 residues or fewer with AlphaFold 2. Each epoch mixes about 16,000 experimental CATH chains with a 10% sample of those predictions, a 1:80 ratio that keeps the larger model from overfitting. Coordinates with pLDDT below 90 (about a quarter) are masked out, each residue's pLDDT enters as a Gaussian radial basis feature, and 0.1 Å of noise is added to the rest. Evaluation uses CATH 4.3 partitioned at the topology level into 16,153 training, 1,457 validation and 1,797 test chains, so test backbones share no topology with training.

On that split the model reaches 4.01 perplexity and 51.6% recovery, against 5.43 and 42.2% for the GVP-GNN state of the art trained on CATH alone. Redesigning the SARS-CoV-2 receptor binding domain, which has no training structure above 0.5 TM-score, recovers 53.6% of the native sequence. On the ACE2-RBD deep mutational scan, log-likelihoods conditioned on all backbone coordinates correlate with measured binding affinity at Spearman 0.64, against 0.03 for sequence-only ESM-1v; masking the receptor binding motif coordinates destroys the correlation, confirming the model reads interface geometry rather than sequence context.

#Applications

The immediate use is fixed-backbone design: handing the model a de novo scaffold, a redesigned enzyme active site or a binder interface and sampling sequences, optionally with partner chains as context. Span-mask training extends this to infilling loops and regions a crystal structure never resolved, and two-conformation conditioning supports multi-state design for switches and biosensors. Because likelihoods are exact, the same checkpoint serves as a structure-aware mutation scorer whenever a wild-type structure is in hand.

#Impact

ESM-IF1 established that machine-generated structures are legitimate training data for protein design, and that model capacity and predicted-data volume have to scale together. Released within months of ProteinMPNN, it became one of the two default inverse folding baselines and a common scoring component inside larger design pipelines. The limits are real: input is capped at 512 residues, only backbone N, Cα and C atoms are read, so ligands and side-chain packing are invisible to it, and sequence recovery is a proxy for foldability rather than a measurement of it. Sampled sequences occasionally degenerate into runs of a single amino acid, and every result in the paper is computational — none of the designs was expressed experimentally.