MoSTERT

Liquid chromatography retention time is one of the strongest orthogonal signals for confirming a peptide identification in mass spectrometry proteomics, and deep learning predictors of it are now standard in rescoring pipelines. They share a blind spot: nearly all tokenize a peptide as a string of amino acid symbols, with each post-translational modification given its own token. A token never seen during training carries no learned meaning, so a peptide bearing an unusual PTM cannot be predicted at all — and few PTMs have enough measured data to train on.

MoSTERT (Molecular Structure Transformer Encoder for Retention Time prediction), from Ceder Dens, Kris Laukens and Wout Bittremieux at the University of Antwerp with Darien Yeung and Oleg Krokhin at the Manitoba Centre for Proteomics and Systems Biology, University of Manitoba, drops the token vocabulary one level down. Each residue — canonical or modified — is rendered as an explicit molecular structure and embedded by a pretrained molecule encoder. Adding support for a PTM the model has never encountered means supplying a SMARTS reaction that describes the chemical change, not collecting a training corpus.

The preprint presents two variants in one framework. Base MoSTERT regresses indexed retention time (iRT) directly from the sequence of molecular embeddings. MoSTERT-2S decomposes the problem: a sequence-level transformer first predicts the iRT of the unmodified peptide, and MoSTERT then predicts only the shift the modifications induce. MoSTERT-2S is the better-performing variant and carries the paper's headline claim.

#Key Features

• Residues as molecules, not tokens: Every amino acid is built as an RDKit molecule and modified in place by a SMARTS reaction, giving a representation that is agnostic to whether a residue is canonical or carries a PTM.
• Extension by chemistry, not data: Covering a new modification requires writing the SMARTS reaction for it. No labelled retention times for that PTM are needed.
• Two-step iRT decomposition: MoSTERT-2S predicts the unmodified peptide's iRT with a token-level transformer, then predicts the PTM-induced shift, combining the accuracy of sequence models on unmodified peptides with molecular-level PTM handling.
• Consensus over training splits: MoSTERT-2S trains one model per training-validation split and takes the median prediction, reducing sensitivity to which PTMs happened to land in validation. The ensemble is fixed at release, not refit on user data.
• Leakage-controlled zero-shot evaluation: Test peptides sharing an unmodified sequence with any training peptide are removed, so reported accuracy reflects genuinely unseen chemistry rather than memorized backbones.

#Technical Details

Per-residue embeddings come from the Molecule Attention Transformer (MAT), a transformer molecule encoder that incorporates interatomic distances and adjacency, pretrained on 2 million molecules sampled from ZINC15. The embedding sequence feeds an 8-layer TAPE-architecture transformer encoder with embedding size 384, trained from random initialization with MAE loss, batch size 1024, and Adam with a 10-epoch warm-up. In MoSTERT-2S, step one is an 8-layer, 96-dimensional TAPE encoder pretrained by masked language modelling and fine-tuned on the training set; step two is a 4-layer, 384-dimensional MoSTERT variant that receives the step-one prediction as an extra input.

Training data derives from the Chronologer corpus of more than 2.6 million peptides, curated down to 2.1 million unique peptides spanning 12 PTMs by removing anomalous iRT values, duplicated measurements, and chemically impossible modifications. Generalization is measured two ways: leave-one-PTM-group-out cross-validation within Chronologer, and an external test set built from the ProteomeTools PTM dataset plus dimethylated peptides from an independent study, retaining only peptides whose PTMs are absent from training. That external set covers 16 unique unseen PTMs. Averaged across 15 PTMs, MoSTERT-2S reaches a mean per-PTM iRT MAE of 10.57 ± 6.57, against 12.18 ± 9.71 for MoSTERT and 12.54 ± 9.17 for DeepLC.

#Applications

The target users are proteomics groups analyzing proteoforms, where modification diversity far exceeds what any training corpus covers. A retention time predictor that extends to arbitrary chemistry lets open or PTM-tolerant searches use retention time as a filtering and rescoring feature for modified peptides, not just for the small set of well-characterized modifications that tools such as Prosit-PTM support. Accuracy varies sharply by modification: monomethylation and glutarylation are predicted well by all evaluated models, while dimethylation, biotinylation and nitrotyrosine remain difficult.

#Impact

MoSTERT reframes peptide property prediction as a molecular rather than a linguistic problem, and the two-step decomposition is a transferable idea — the same split could apply to collision cross section or fragment intensity prediction, where the unmodified peptide is well modelled and the modification is the hard part. One practical caveat dominates: the preprint contains no code or data availability statement, and no repository, weights, or hosted inference path accompanies it, so the models described here cannot currently be run by a reader. It remains a preprint under a CC-BY-NC license, and parameter counts are not reported for either variant.