Loading…
Structure-aware protein language model pairing amino acid tokens with Foldseek 3Di structural states, outperforming ESM-2 across 10 downstream tasks.
SaProt is a protein language model developed at Westlake University that addresses a fundamental limitation of sequence-only protein language models: the inability to directly encode three-dimensional structural information during pre-training. Rather than relying solely on amino acid tokens, SaProt introduces a structure-aware vocabulary that pairs each residue's one-letter amino acid code with a corresponding structural state token derived from Foldseek's 3Di alphabet. This dual-token representation allows the model to learn the joint language of protein sequence and structure simultaneously, rather than treating structure as a secondary prediction target.
The 3Di tokens are generated by running Foldseek on experimental PDB structures or AlphaFold 2 predictions, encoding local backbone geometry and side-chain orientation into one of 20 discrete structural states. Each residue in the input is represented as a two-character token — for example, "Ac" where "A" is alanine and "c" is the corresponding 3Di state. Regions with low AlphaFold 2 confidence (pLDDT below 70) are masked with a placeholder character, keeping noisy structural assignments out of the pre-training signal. This lets SaProt run on any protein with a known or predicted structure — in practice, the entire AlphaFold Database.
SaProt was presented as a spotlight paper at ICLR 2024, one of the most competitive machine learning venues, signaling broad recognition of its methodological contribution. The work was led by Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan. A follow-on ecosystem, SaprotHub, was posted as a preprint in 2024 and published in Nature Biotechnology in 2025 as "Democratizing protein language model training, sharing and collaboration," with an author list that grew well beyond the original team to include Martin Steinegger, Sergey Ovchinnikov and the Open Protein Modeling Consortium (OPMC). It provides no-code fine-tuning and model-sharing infrastructure on top of the SaProt foundation, and distributes community-contributed fine-tuned adapters rather than new pretrained checkpoints.
SaProt's 650M parameter flagship model shares its architecture with ESM-2 — a transformer encoder trained with masked language modeling — but extends the tokenizer to accommodate the paired sequence-structure vocabulary. Pre-training ran in two phases: roughly 40 million AlphaFold2-predicted structures covering known sequence space, then about 60,000 experimental PDB structures to ground the model in empirical data. The 1.3B variant extends further, adding 200 million OMG_prot50 sequences and 150 million NCBI sequences filtered at 70% identity. Training the 650M model required 64 NVIDIA A100 80GB GPUs running for approximately three months.
The 650M PDB-trained checkpoint sits consistently above same-size ESM-2 baselines: Spearman's rho on thermostability 0.680 → 0.724, Fmax on enzyme classification 0.868 → 0.882, subcellular localization accuracy 82.09% → 85.57%. Contact prediction gains are especially pronounced, reflecting the structural information encoded in the 3Di tokens. At inference, Foldseek generates structural tokens on the fly from any PDB or CIF file, and the 1.3B model performs competitively on amino acid tokens alone, so it applies to proteins without structures.
SaProt suits researchers working on protein function annotation, variant effect prediction, and protein engineering. Clinical researchers can apply its zero-shot mutation scoring to prioritize variants of uncertain significance in ClinVar-style databases, and protein engineers can predict thermostability changes, enzyme activity, and binding affinity shifts without labeled training data for each new protein family. Wet-lab groups without ML expertise can fine-tune task-specific models through SaprotHub's ColabSaprot interface, which packages LoRA-based fine-tuning into a point-and-click workflow on free-tier Colab hardware and lets the resulting adapters be shared back to the hub. The model is also a natural starting point for structural bioinformatics pipelines combining Foldseek structure search with sequence-based machine learning.
SaProt is a conceptually important step toward multimodal protein representations, demonstrating that integrating structure directly into the input vocabulary — rather than through auxiliary supervision or post-hoc structure modules — yields consistent gains across diverse tasks. Its ICLR 2024 Spotlight designation reflects peer recognition in the broader machine learning community, not only within computational biology. The SaprotHub follow-on has extended the model's reach by removing the technical barriers to fine-tuning and sharing. Two limitations are worth noting: the 35M and 650M models depend on 3Di tokens for best performance, requiring AlphaFold2 predictions or experimental structures for every protein of interest, and the models a user finds on the hub are community contributions of unvetted provenance rather than the audited SaProt base checkpoints.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.