Loading…
Inverse protein folding from backbone coordinates, chaining a pretrained structure encoder into a pretrained sequence autoencoder on small data.
No providers recorded yet. Browse all providers
Inverse folding models learn from pairs of a protein backbone and the sequence that folds into it, and those pairs are scarce: the standard CATH 4.2 split supplies 18,204 training structures, while the sequence databases available for pretraining hold orders of magnitude more. The usual answers attack the shortage from the data side — a stronger equivariant encoder squeezing more out of the same pairs, or manufactured pairs from folding millions of sequences with a structure predictor, as ESM-IF did with roughly 12 million predicted structures. Neither brings in the prior knowledge already sitting on the sequence side.
MMDesign takes the other route: it assembles the design network from parts that are already trained. A GVPConv structural module reads the backbone's N, Cα and C coordinates, its parameters taken from the off-the-shelf ESM-IF structure model and initialized as in the same group's CCPL work, where a GVPConv encoder is pretrained against a large protein language model through contrastive alignment. Its output feeds an encoder-decoder Transformer pretrained as a sequence-to-sequence autoencoder. Only then are the two concatenated and trained together, once, on CATH 4.2, with a cross-layer alignment loss keeping structural features consistent with contextual ones throughout.
The model was built by Jiangbin Zheng and Stan Z. Li at the AI Lab of Westlake University with Zhejiang University, posted in December 2023 and presented as an oral at IEEE ICME 2024. It is the earliest entry in the group's cross-modal inverse folding line, which continues in CrossDesign. An unrelated nanobody design model from Beijing Molecule Mind shares the name MMDesign.
The structural module is four GVPConv layers with scalar and vector node dimensions of 1024 and 256, a 30-nearest-neighbour graph and 0.1 dropout. The contextual module keeps the eight encoder and eight decoder layers of the pretrained autoencoder — eight attention heads, embedding dimension 512, learned positional embeddings — and discards its embedding layers when transferred. That autoencoder is pretrained on the sequences of the CATH training split only, as a sequence recovery task, so its prior semantic knowledge is in-domain rather than drawn from a large sequence corpus. Training uses SGD at learning rate 1e-3 with batch size 5 on a single NVIDIA A100 80GB; the parameter count is not reported.
On the CATH 4.2 test set the model reaches 3.86 perplexity and 54.88% sequence recovery, against 45.96% for ProteinMPNN, 51.66% for PiFold, and 52.51% for ESM-IF, which trained on CATH plus AlphaFold DB. Recovery is 39.89% on the short subset and 39.24% on single chains. The same checkpoint applied to the out-of-domain Ts50 and Ts500 sets gives 59.87% and 61.06% recovery at perplexity 2.89 and 3.20. An ablation isolates the two priors: 36.86% recovery with neither module pretrained, 41.81% with only the contextual module, 49.41% with only the structural module, and 54.88% with both; the structural prior contributes more, but the contextual one is worth five points alone.
The task is fixed-backbone sequence design: given a target fold, propose sequences expected to adopt it, for de novo design, scaffold redesign or stabilization campaigns. The practical appeal is for groups without the compute or the augmented corpora that large-data inverse folding models assume — the whole training run fits on one GPU. More broadly, pretraining both modalities and then transferring them is a template for cross-modal protein tasks where paired supervision is thin but single-modality pretraining is abundant.
MMDesign's lasting contribution is the demonstration that a small paired dataset suffices when both ends of the network already carry priors, with the ablation as its cleanest evidence: the pretrained contextual module helps even when the structural module is untrained. The same group carried the approach forward into CrossDesign, which replaces the transferred autoencoder with a frozen protein language model used as a training-time teacher. The limitations are worth stating plainly. No code or weights have been released, and neither version of the paper carries an open license — the preprint sits under arXiv's default distribution terms and the conference version under IEEE copyright — so the results cannot be independently reproduced or the model run by others; the parameter count is unreported; and evaluation is entirely in silico, resting on native sequence recovery and perplexity, with no experimental validation of any designed sequence.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.