Structure-aware protein language model aligning sequence and 3D structure by contrastive learning, with adapter and LoRA fine-tuning tools.
No providers recorded yet. Browse all providers
S-PLM is a structure-aware protein language model from the Xu lab at the University of Missouri, with a collaborator at the University of Kentucky. It addresses a practical asymmetry in protein representation learning: models that fuse sequence and structure generally outperform sequence-only models, but they need a structure at inference time, which means running a folding predictor for every query. S-PLM instead pushes structural information into the sequence encoder during pretraining, through multi-view contrastive learning that aligns sequence and structure embeddings in a shared latent space. After pretraining, only the sequence encoder is used, so inference takes amino acids alone.
The April 2025 preprint introduces S-PLM2, the second generation of the method. The original model, published in Advanced Science, encoded structure as Cα contact maps processed by a Swin Transformer and aligned them with sequence embeddings at the whole-protein level. S-PLM2 replaces that with a Geometric Vector Perceptron operating directly on 3D backbone coordinates, and extends contrastive alignment to the residue level, which matters for per-residue tasks like secondary structure prediction. Both versions are distributed and benchmarked side by side.
The second half of the work is a fine-tuning toolbox. Because the encoder inherits ESM-2's parameter count, full fine-tuning is impractical for most downstream tasks, so the repository ships top-layer tuning, adapter tuning, and LoRA behind a common configuration interface and evaluates all three across five prediction tasks.
The sequence encoder is ESM-2 t33 650M, producing 1,280-dimensional per-residue embeddings, with Structure-Aware Modules inserted into the top transformer layers; residue embeddings project to 64 dimensions and pooled protein embeddings to 256 for the contrastive objective. The structure branch is a Swin Transformer over continuous contact maps in S-PLM1 and a GVP graph network over N, Cα, C, and O coordinates in S-PLM2, with optional Foldseek-derived scalar and vector node features. Pretraining used 500,000 Swiss-Prot sequences with AlphaFold DB structures, plus 41,500 held out for validation, truncated at 512 residues and trained on a single A100 for over 10,000 steps.
On unsupervised clustering, S-PLM2's Calinski-Harabasz index on CATH hierarchies runs roughly 140% above ProstT5 at the class level and about 50% higher at architecture and topology. Clustering 336 human kinase domains into nine groups gave an adjusted Rand index of 0.78 for S-PLM2 and 0.72 for S-PLM1, against 0.61 for ProstT5 and 0.28 for ESM-2; on 242 deaminases the corresponding figures were 0.86 and 0.87 against 0.80 and 0.63. On supervised tasks, S-PLM2 leads GO biological process (Fmax 0.498) and CB513 secondary structure (accuracy 0.88), while S-PLM1 leads GO molecular function (Fmax 0.685); EC number prediction reaches Fmax 0.888, close to ESM-GearNet at 0.890 and PST at 0.897 — both of which require structure as input, as does SaProt.
The model suits anyone who needs structure-informed protein representations without a structure pipeline: enzyme family and function annotation, fold and enzyme reaction classification, GO and EC prediction, secondary structure prediction, and unsupervised clustering of protein families where sequence identity alone fails to separate functional groups. Because tuning is parameter-efficient, a lab with a modest labelled dataset and one GPU can adapt the encoder to a new task without touching the backbone.
S-PLM's argument is that structural knowledge can be distilled into a sequence encoder rather than supplied as a second input, and its benchmarks support the weaker but useful version of that claim: it matches or approaches methods that consume structures directly while asking for none. The gains over the base ESM-2 encoder are real but uneven — under matched fine-tuning budgets S-PLM1 beat or matched ESM-2 in 17 of 27 comparisons and S-PLM2 in 16 of 27 — so the advantage is task-dependent rather than uniform. Pretrained weights for both versions are distributed via institutional SharePoint links rather than a model hub, which complicates programmatic access, and the 2025 preprint has not been peer reviewed, though the underlying V1 method has been. The repository is MIT licensed.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.