Loading…
Protein language model trained from scratch on MD and normal-mode dynamics, representing residue fluctuation and co-movement from sequence.
Most protein language models — ESM2 being the canonical example — are trained to reconstruct masked residues from sequence context. That captures evolutionary signal, but not the conformational motion that underlies protein function: two sequences with identical conservation profiles can behave very differently in solution.
SeqDance, from Chao Hou, Haiqing Zhao, and Yufeng Shen at Columbia University, replaces that objective outright. It is supervised on biophysical dynamics — per-residue and pairwise descriptors of fluctuation and co-movement, derived from molecular dynamics trajectories and normal-mode analysis of tens of thousands of proteins — and it is trained from random initialization, so nothing evolutionary or structural is inherited. The question it is built to answer is whether motion alone is a sufficient training signal for a protein language model.
Its companion ESMDance takes the opposite stance on the same data, keeping a frozen ESM2-35M backbone and learning to map its representations onto the dynamic properties. Reading the two together isolates what the dynamics objective contributes on its own. The work was posted to bioRxiv in October 2024 and published in PNAS as "Protein language models trained on biophysical dynamics inform mutation effects."
The architecture is a Transformer encoder identical in shape to ESM2-35M: 12 layers, 20 attention heads per layer, an embedding dimension of 480, and roughly 35 million parameters. Only the initialization and the objective differ.
Training data are dynamic biophysical descriptors derived from MD trajectories and normal-mode analyses, combining sets spanning on the order of 64,000–65,000 proteins, with MMseqs2 deduplication applied. The full corpus of sequences and extracted dynamic properties — around 100 GB covering 64,403 proteins — is published alongside the model, together with train-test splits and an independent dynamicPDB test set.
The model supplies representations and predictions of conformational properties for researchers studying protein motion and stability. Radius of gyration and dynamic contact predictions apply to ordered and disordered proteins alike, which matters for intrinsically disordered regions that fold-prediction methods characterize poorly. The framing is complementary to ensemble samplers and structure predictors: where AlphaFold-style models predict a static fold and methods like AlphaFlow or BioEmu sample conformational ensembles, this embeds dynamic behavior directly into a sequence representation that can be queried without running a simulation.
The value of training from scratch here is diagnostic as much as practical. Because the companion model differs only in starting from a pretrained backbone, the pair together establishes that the dynamics objective carries information beyond conservation rather than merely re-encoding it — a claim that a single dynamics-trained model could not have supported on its own. Peer-reviewed publication in PNAS, GPL-3.0 code, MIT weights, and a CC-BY-4.0 release of the entire training corpus make the result unusually reproducible. At 35M parameters the model is modest in scale, and the obvious next step is applying the dynamics objective to larger backbones and broader protein families.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.