Loading…
Protein structure tokenizer that encodes all-atom folds as artificial amino acids and decodes them back to coordinates at TM-score above 0.96.
No providers recorded yet. Browse all providers
An enzyme's active and inactive conformations share one amino-acid sequence, so a one-dimensional representation is blind to the difference between them; the 3D coordinates that do capture it carry SE(3) symmetry and peptide-geometry constraints a generative model must be built around rather than simply consume. ProTokens closes the gap with a second alphabet: a vocabulary of "artificial amino acids" in which each residue of a folded structure gets one discrete token, and the token string decodes back to atom coordinates.
The construction rests on a probabilistic tokenization argument: a distribution of function-related structures factors into a discrete part — which metastable state the protein occupies — and a continuous part covering fluctuations within it. Metastable states are countable at a given observation timescale, which makes the discretization principled, and natural amino acids are the coarse case of the same construction at the folding timescale — so the learned vocabulary contains Anfinsen's twenty tokens rather than competing with them.
The vocabulary is extracted by an autoencoder the authors call the ProToken Distiller, developed at Changping Laboratory with the Gao group at Peking University and Soochow University. Its objective conjugates two classical problems: the encoder performs inverse folding with the fixed amino-acid alphabet replaced by a learnable codebook, and the decoder performs structure prediction from whatever that codebook emits, with gradients flowing end to end and structures alone supplying supervision. The preprint appeared in November 2023 and was revised three times through October 2024; the last revision extended a backbone-only vocabulary to all atoms, added multimer benchmarks, and released code, checkpoints and a token-compressed copy of the PDB. Downstream work spells the tokenizer itself "ProToken", in the singular.
The Distiller stacks an encoder, a variational quantization module and a generative decoder in JAX and Flax. Training used a cleaned single-chain subset of the PSP dataset drawn from the Protein Data Bank with a truncation date of 13 October 2021, prior to CASP14, excluding chains with unresolved gaps, chains under 30 residues and NMR structures: roughly 552,000 chains cropped to 256 residues. The backbone codebook was initialized at 768 entries and reduced to 513 by variational clustering, one of them the wildcard, while the 20 side-chain tokens were fixed. Backbone and side-chain embeddings compress to 32 and 8 dimensions, for a 40-dimensional ProToken embedding.
On held-out CASP14 (87 targets) and CASP15 (44 targets) single chains, reconstruction from tokens reaches an average TM-score above 0.96 and a median above 0.97 with the decoder run deterministically. Fidelity is largely insensitive to chain length and shape but tracks stability: it degrades on extremely extended folds, and at residue resolution falls as crystallographic B-factors rise and as AlphaFold 2 pLDDT falls. Despite single-chain training, the tokenizer generalizes to high-confidence dark clusters of the AlphaFold Database with no statistical loss of accuracy, and to disconnected multi-domain assemblies and real homo- and heterodimers, where interface quality averages a DockQ above 0.49.
The tokens are useful wherever structure has to be stored, searched, or fed to a model that expects a sequence: databases can be compressed and indexed as token strings, and generative models can emit structures as tokens or embeddings without handling rotational symmetry themselves. That last use is where the model has been adopted most. ProtTeX vendors the tokenizer as the structure front-end of a Llama-3-based protein language model, and AminoAseed wraps it as one of nine pretrained tokenizers in a comparative evaluation of structure vocabularies.
ProTokens is among the earliest structure tokenizers to argue for discretization from protein physics rather than convenience, and that framing — with the wildcard token that folds amino acids into the same vocabulary — is the part most likely to outlive any particular codebook. It has become a reference point for the tokenizers that followed: AIDO.StructureTokenizer and ProteinAE both benchmark reconstruction against it. Two things bound its use. The work remains a preprint under a CC-BY-NC-ND license with no peer-reviewed version, and Changping Laboratory states in the paper that a provisional patent application on the discretization of biopolymer 3D structures is pending. And the fidelity analysis is candid about where tokens fail: disordered and marginally stable regions are what the vocabulary smears.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.