ProteinDPO

An inverse-folding model asked to write a sequence for a given backbone has never seen a stability measurement. It learned which residues sit comfortably in which local geometry, and because the Protein Data Bank skews toward proteins that survived expression and crystallization, its sequence likelihoods track stability by side effect rather than by design. That side effect is what the authors of ProteinDPO name the alignment gap: the rules absorbed during unsupervised pretraining are not the property anyone wants optimized.

The obvious repair is to fine-tune on the stabilizing variants, which makes the model sharper inside the training distribution and duller outside it. ProteinDPO takes the other route, applying direct preference optimization — the algorithm used to align language models to human preference labels — to ESM-IF1. Each training example gives the model a native backbone as the prompt and a pair of variants as the response: one measured to be stabilizing, one destabilizing. Because the destabilizing member contributes gradient too, the objective sees the shape of the fitness landscape instead of a filtered list of good sequences, and a KL penalty against the frozen pretrained model preserves the structure-conditioned grammar while the stability signal is written in.

The work comes from Talal Widatalla and colleagues in Brian Hie's group at Stanford University and the Arc Institute, with a collaborator at UCSF. A May 2024 preprint preceded the Nature Methods paper, which added the wet-lab hemagglutinin campaign.

#Key Features

• Preference pairs rather than filtered positives: Training consumes both members of a stabilizing/destabilizing pair for one backbone, which is what separates it from supervised fine-tuning on stable variants alone.
• Three alignment objectives: The paper derives paired (Bradley–Terry), ranked (Plackett–Luce) and a new weighted objective that consumes the scalar ΔΔG directly instead of discretizing it into rankings. Only the paired-objective checkpoint is released.
• One checkpoint scores and generates: The same frozen weights rank point mutants by likelihood and autoregressively sample whole stabilized sequences onto an arbitrary backbone, with a --fixed_pos argument to hold conserved residues in place.
• Transfer far outside the alignment set: Aligned only on ΔΔG of one- and two-substitution variants in 40–72 residue monomers, it improves scoring of multichain binding affinity, antibody thermal melting and stability of proteins an order of magnitude larger.

#Technical Details

The base model is ESM-IF1, a 142M-parameter architecture pairing a geometric vector perceptron encoder over backbone coordinates with an autoregressive transformer decoder; DPO alignment changes the weights, not the parameter count. The alignment data is the Megascale cDNA display proteolysis dataset, curated to roughly 660,000 variants across 403 protein domains and clustered with Foldseek easy-cluster at a 0.5 similarity threshold, then split 90/5/5 by cluster so evaluation folds stay structurally unseen. Training ran on a single H100 at learning rate 1×10⁻⁷.

On the Megascale holdout, per-protein correlation between likelihood and ΔΔG reaches Pearson R 0.72–0.73 and Spearman ρ 0.69–0.72 across the three objectives, against 0.55/0.53 for vanilla ESM-IF1 and 0.59/0.57 for the supervised fine-tune, with stabilizing-versus- destabilizing AUROC of 0.82–0.84 versus 0.74. On S669, scored by a model retrained on a Megascale set stripped of S669 homologs, it reaches Pearson R 0.44–0.47 where the supervised ThermoMPNN scores 0.43; on the homolog-free FireProt curation it trails ThermoMPNN (0.60 against 0.65). Zero-shot binding-affinity correlation on SKEMPIv2 and AB-Bind rises 0.02–0.08 Pearson over the unaligned model, and ranking of thermal melting points across 483 clinical and human-derived antibodies rises 0.08–0.12 — regimes those supervised baselines cannot address, since they score single chains and single mutations. Sequences sampled onto the 1STN, 1PHP and 4P79 backbones score lower Rosetta cart_ddg energy than the natives while still folding back to them (ESMFold pLDDT above 80, scTM 0.96–0.99).

#Applications

The concrete use is stabilizing a protein you already have without training a predictor for it first. Applied to the metastable prefusion conformation of H5N1 hemagglutinin from a 2004 Vietnam strain, ProteinDPO ranked all single substitutions and recovered six of eight stabilizing mutations previously found by structure-guided design; of 45 total designs, 27 raised the melting temperature, by up to 17 °C, and the same substitutions transferred to 2024 dairy-cattle and British Columbia H5N1 strains for gains of 13 °C and 32 °C, with binding to the broadly neutralizing antibodies CR6261 and 13D4 preserved. The same workflow covers antibody developability triage and enzyme thermostabilization.

#Impact

The durable claim is the generalization rather than any leaderboard position: alignment on one cheaply measured property, in one narrow structural regime, moved performance on properties and size scales the alignment data never contained, while supervised fine-tuning on the same data overfit. That makes preference optimization a general recipe for writing experimental measurements into a pretrained biological generative model, which is how the authors frame it. Three caveats travel with the release: only the paired-objective weights are published, inference also requires Meta's original ESM-IF1 checkpoint, and the repository warns that stabilizing a backbone carries no guarantee of preserving its function, so conserved positions must be fixed by hand. Retained antibody binding is evidence of preserved epitopes, not of vaccine immunogenicity.