Protein Feature Token Classifiers

The ACMG/AMP guidelines let a clinical geneticist call a missense variant moderately pathogenic when it lands in a mutational hotspot or a critical functional domain — the PM1 criterion. Applying it requires knowing where those domains are, and UniProtKB/Swiss-Prot curates only a fraction of the human proteome feature by feature: a variant in an unannotated stretch of an uncharacterized protein simply cannot be assessed under PM1, no matter how disruptive it is. Pathogenicity scores fill part of the gap, but a single number says nothing about the mechanism a substitution breaks.

Ali Saadat and Jacques Fellay at EPFL, with the SIB Swiss Institute of Bioinformatics and the Precision Medicine Unit at Lausanne University Hospital and the University of Lausanne, recast the problem as per-residue token classification. Instead of scoring pathogenicity, they fine-tune a protein language model to answer, for every amino acid in a sequence, whether it carries each of 20 Swiss-Prot feature annotations — active site, binding site, DNA binding site, disulfide bond, transmembrane region, β-strand, α-helix, turn, domain, motif, repeat, zinc finger and eight others. Passing the reference and the alternate protein sequence through the same classifiers and subtracting the two probability profiles gives a residue-level readout of which features a variant gains or loses.

Released as an arXiv preprint in October 2024 and published in Computational and Structural Biotechnology Journal in May 2025, the family is built on ESM-2 at five sizes and on the ProtT5 encoder from ProtTrans. It is a complement to pathogenicity predictors such as AlphaMissense rather than a competitor: the output is a mechanism, not a score.

#Key Features

• Mechanistic variant readout: The difference between predicted feature probabilities for the wild-type and mutant sequence identifies, position by position, which structural or functional feature a substitution disrupts — the kind of hypothesis a wet-lab assay can be designed around.
• One classifier per feature: Annotation sparsity ruled out a single multi-task head, so each of the 20 features gets its own fine-tuned model with a fully connected binary classification head over the language model encoder.
• Proteome-wide reannotation: The classifiers are run over proteins Swiss-Prot leaves uncurated for a given feature. Predicted residues match curated ones on GERP conservation and REVEL pathogenicity distributions for most features, with DNA binding sites, zinc fingers, modified residues and repeats the exceptions.
• Evidence for PM1 at scale: Fisher's exact tests over 46,504 pathogenic and 72,150 benign ClinVar/gnomAD missense variants identified 13 features enriched for pathogenic variants and seven enriched for benign ones.
• Parameter-efficient fine-tuning: Rank-4 LoRA adapters on the attention projections keep most pretrained weights frozen, so the full grid of six backbones by 20 features is trainable on a single GPU.

#Technical Details

Training used 20,434 human proteins from UniProtKB/Swiss-Prot, clustered with MMseqs2 at 20% coverage and 20% sequence identity into 7,538 clusters and split 70/15/15 at the cluster level to limit leakage between train and test. Backbones span ESM-2 at 8M, 35M, 150M, 650M and 3B parameters plus ProtT5 (prot_t5_xl_uniref50, a 1.2B-parameter encoder-decoder). Each model carries a fully connected head of hidden size 32 for per-residue binary classification; LoRA is applied at rank 4 with α = 1 to the query, key, value and output projections. Training ran for 10 epochs at a learning rate of 3 × 10⁻⁴ with dropout 0.2 and cross-entropy loss on one 40 GB Nvidia A100, keeping the lowest-validation-loss checkpoint; inference fits on a 24 GB A10G. A frozen-embedding classifier — the same head trained on unmodified embeddings — serves as the control that isolates what fine-tuning contributes, and improved AUROC for most model-feature combinations. The fine-tuned ESM2-3B models were selected for every downstream analysis.

#Applications

Applied to gnomAD, the enriched-feature map supplied PM1 evidence that reclassified 110,304 of 1,692,568 variants of uncertain significance (6.5%) as pathogenic. On 771 curated ClinGen variants across 54 genes, pathogenic substitutions disproportionately disrupted active sites, disulfide bonds and functional domains, while benign ones perturbed tolerant features such as compositional bias and signal peptides. Two worked examples show the granularity: DCLRE1C p.His35Asp loses domain annotations at several positions and gains a binding site, and RUNX1 p.Ala134Pro alters turns, β-strands, α-helices and DNA binding annotation.

#Impact

Code, checkpoints and processed data are all public, and the per-feature checkpoints are small enough to run in an ordinary variant-curation pipeline, which makes the framework usable by clinical labs without ML infrastructure. The authors are explicit about the limits: predictions in poorly characterized regions still require experimental validation, some features were dropped for weak performance, no hyperparameter search was performed, and the variant sets inherit ClinVar's bias toward clinically studied genes and gnomAD's enrichment for European ancestry. Reclassification here is computational evidence contributed to an ACMG/AMP assessment, not a diagnostic call.