Protein language model interpretability adapter that factors ESM2 and ProtBERT embeddings into named biochemical features plus a residual subspace.
No providers recorded yet. Browse all providers
Protein language models produce embeddings that work extremely well as features and explain almost nothing. An attribution method applied to a PLM vector can tell you which coordinates mattered, but those coordinates have no biological names, so the explanation stops at the numbers. PLM-eXplain (PLM-X) attacks the problem at the representation rather than the attribution stage: instead of interpreting the embedding after the fact, it reorganises the embedding space so that part of it is interpretable by construction.
PLM-X is an adapter that maps a frozen PLM embedding onto two complementary subspaces. The first is a small interpretable subspace whose 34 axes are tied to established biochemical descriptors — amino-acid composition, three- and eight-state secondary structure, accessible surface area, and hydropathy-related properties. The second is a residual subspace that carries whatever predictive signal the named features do not capture. A decoder reconstructs the original embedding from the two halves together, so the transformation is information-preserving rather than a lossy projection onto handcrafted features. Downstream classifiers are then trained on the transformed embedding, and feature attribution over the interpretable axes yields statements about secondary structure or hydrophobicity instead of about coordinate 217.
The adapter was developed by the AI Technology for Life group at Utrecht University, posted as a preprint in April 2025 and published in Bioinformatics in 2026. It is trained once, on a fixed structural corpus, and then reused across tasks without retraining — a different bet from sparse-autoencoder approaches such as PLM-SAE, which discover unsupervised features rather than aligning axes to a predefined biochemical vocabulary.
The adapter is a dual-branch encoder-decoder. Two parallel MLP encoders map the PLM embedding to the interpretable and residual subspaces; a decoder reconstructs the input from their concatenation; and a discriminator sits behind a gradient reversal layer on the residual branch. Training combines three objectives: a supervised loss on the interpretable axes (cross-entropy for the secondary-structure classes, binary cross-entropy for binary descriptors, L1 for continuous ones), the adversarial term, and a reconstruction loss. Supervision comes from 20,298 human protein structures drawn from the AlphaFold Protein Structure Database, with secondary structure and solvent accessibility computed by DSSP and sequence descriptors by Biopython. Reconstruction reaches a mean absolute error of 0.068.
Evaluation covers three protein-level tasks with either a pooled XGBoost classifier or a per-residue CNN on top of the fixed transformed embedding: transmembrane helix prediction on DeepTMHMM data (ROC-AUC 0.99, F1 0.93), aggregation propensity on WALTZ-DB 2.0 (ROC-AUC 0.90, F1 0.80), and extracellular vesicle association across the human proteome (ROC-AUC 0.79, F1 0.62). ProtBERT-bfd backbones track closely, at ROC-AUC 0.98, 0.88 and 0.78 respectively. A control using only the handcrafted descriptors performs worse on every task, which is the evidence that the residual subspace carries real signal rather than noise.
PLM-X suits any group already building classifiers on frozen PLM embeddings that needs to justify predictions biologically — membrane protein annotation, aggregation risk in protein engineering and biologics formulation, and biomarker work such as extracellular vesicle cargo prediction. Because the adapter is fixed, adopting it costs one embedding transformation pass plus the downstream training the group was already doing, and model behaviour can then be discussed in terms of secondary structure and hydropathy with collaborators who do not read embeddings.
The work sits in a growing effort to make PLM representations legible rather than merely accurate, and it is unusual in showing that imposing a named coordinate system costs no measurable accuracy across three independent tasks. Its scope is bounded: the adapter is supervised by structural annotations from human AlphaFold DB models, so its interpretable axes reflect that distribution, and the demonstrated tasks are protein-level binary classification. The public repository contains the adapter training code, the XGBoost and CNN downstream pipelines, and a trained downstream classifier checkpoint, with embeddings, task data and model files distributed through a linked Google Drive folder; no license file accompanies the code, and no model card or data card is published beyond the paper.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.