Loading…
Multimodal discrete-diffusion protein language model that co-generates amino acid sequence and 3D backbone structure from a single transformer.
No providers recorded yet. Browse all providers
Designing a protein has usually meant running two models back to back: a structure generator proposes a backbone, then a separate inverse-folding network such as ProteinMPNN writes a sequence for it. The sequence is fitted to the structure after the fact, so nothing in the pipeline represents the joint distribution over the two, and a designed pair's compatibility can only be checked afterwards, never modeled.
DPLM-2 collapses that cascade into one discrete-diffusion transformer that treats sequence and structure as two token streams over the same residues. Backbone coordinates become discrete structure tokens through a learned tokenizer, take the positional encodings of the corresponding amino acids, and are concatenated into a single input; the model denoises both streams under an absorbing discrete-diffusion objective with an independent noise schedule per modality. One set of weights therefore carries the joint distribution and all of its marginals and conditionals, so co-generation, folding, inverse folding and motif scaffolding become different masking patterns at inference rather than different models.
DPLM-2 was developed by ByteDance Research with the Department of Computer Science at Nanjing University, posted as a preprint in October 2024 and published at ICLR 2025. It extends the sequence-only Diffusion Protein Language Model (DPLM), which is not currently in this catalog, and is refined by DPLM-2.1, an ICML 2025 follow-up that identifies structure-token prediction as this model's weakest link; the three share one repository but are distinct checkpoints. Its closest contemporary is ESM3, which also tokenizes structure into a language model but pretrains multimodally from scratch and generates one modality after another.
DPLM-2 is released at 150M, 650M and 3B parameters as bidirectional ESM-style transformers; the 650M checkpoint has 33 layers, hidden size 1280 and 20 attention heads, over a vocabulary of 8,229 amino acid, structure and special tokens. Training uses ~20K clustered experimental PDB structures plus ~200K AlphaFold-predicted SwissProt structures from the AFDB, capped at 512 residues. On unconditional co-generation the 650M model reaches scTM 0.925 and scRMSD 3.90 Å at mean pLDDT 82.7, matching a MultiFlow checkpoint that depends on ProteinMPNN distillation, and its secondary-structure composition tracks natural PDB proteins more closely than RFDiffusion's. Folding is zero-shot: 5.67 Å RMSD and 0.83 TM-score on a PDB date split at 650M, 4.54 Å and 0.86 at 3B, and 3.15 Å after folding fine-tuning, against ESMFold at 2.84 Å and 0.93. Inverse-folding amino acid recovery on the same split is 54.8% at 650M and 61.7% at 3B, above ESM3's 49.5%. As a representation model, DPLM-2 650M raises thermostability Spearman ρ to 0.714 and EC Fmax to 0.878 over sequence-only ESM-2 and DPLM, while trailing SaProt on most predictive tasks. Despite the 512-residue cap, sampled sequences hold their pLDDT out to 1,000 residues.
DPLM-2 suits design campaigns where backbone and sequence must be produced together rather than in series — de novo generation of designable, natural-looking folds, scaffolding of a functional motif whose chemistry matters as much as its geometry, and structure-conditioned redesign. The same weights fold candidates for in-silico screening and supply structure-aware embeddings for property predictors, so one 650M model on a single GPU covers a whole design-and-triage loop. Apache-2.0 code and checkpoints also make it a practical base for further fine-tuning.
DPLM-2 established that a moderate structure corpus plus a public pretrained sequence language model is enough to build a multimodal protein foundation model, rather than the from-scratch multimodal pretraining at far larger scale and under restrictive licensing that ESM3 required. The discrete structure-token recipe behind it was carried forward by DPLM-2.1 and later token-based protein models. The authors are candid about the limits: sparse structure data constrains representation quality, discretization discards atomic detail and interatomic distances, modeling stops at backbone atoms of single chains under a 512-residue training length, and every result is computational, with no experimental characterization of the designs.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.