KaML-ESM2 & KaML-ESMC

The pKa of a titratable side chain determines whether it is charged at physiological pH, and therefore whether it can act as a catalytic acid, a base, a nucleophile, or a salt-bridge partner. Predicting those values has been a structure-based problem for decades: Poisson-Boltzmann solvers, empirical schemes such as PROPKA3, constant-pH molecular dynamics, and machine learning models trained on structures. All of them need a structure, and all are sensitive to which structure you give them.

KaML-ESM2 and KaML-ESMC, from Jana Shen's lab at the University of Maryland School of Pharmacy, drop the structure requirement. They are small multilayer perceptron task heads trained on the frozen per-residue embeddings of ESM-2 and ESM Cambrian, predicting the pKa shift of Asp, Glu, His, Cys, Tyr, and Lys residues directly from sequence. The underlying claim is a biological one: because protein language models learn from sequences shaped by hundreds of millions of years of selection, their representations encode not only structure and function but, indirectly, electrostatic character.

The models are the sequence-based successors to the lab's structure-based KaML family, which included the gradient-boosted-tree KaML-CBT and graph attention network KaML-GAT. First posted as a preprint in April 2025 and revised since, the work ships as an end-to-end platform: a command-line tool, a hosted web server, and a precomputed pKa atlas covering the human proteome.

#Key Features

• Sequence-only prediction: No experimental or predicted structure is needed, making proteome-scale application feasible for targets that lack structures or whose predicted models contain large low-confidence regions.
• Frozen backbone, thin head: Parameter-efficient fine-tuning of the backbone with qLoRA gave a negligible net change in error, so the released models train only the task heads on frozen embeddings.
• GAINES data augmentation: An augmentation strategy named auGment dAta wIth lateNt spacE Sampling retrieves residues from a protein database whose embeddings closely match a labelled query residue and transfers the experimental pKa label, addressing the scarcity of measured values.
• Separate acid and base channels: Independent heads handle acidic (Asp, Glu, Cys, Tyr) and basic (His, Lys) residues, each deployed as an ensemble of 500 models to give per-prediction standard errors.
• Human proteome atlas: A browsable set of precomputed predictions covering 1,877,704 sites in 18,192 human proteins accompanies the code.

#Technical Details

Each task head is a four-layer MLP with three hidden layers, taking per-token embeddings from ESM-2 650M or the 6B-parameter ESM Cambrian model and regressing the pKa shift relative to model-peptide reference values. Layer choice matters: a head on first-layer ESM-2 representations gave an RMSE of 1.22 versus 0.75 from the second-to-last layer. Training used the PKAD-3r experimental database expanded by GAINES, with 50 stratified random holdout splits grouped by unique residue. Across those holdouts the GAINES-trained models reached overall RMSEs of 0.48 ± 0.02 (KaML-ESM2) and 0.53 ± 0.02 (KaML-ESMC) with Pearson correlations of 0.98 and 0.97. Partitioning the data with CD-HIT at 70% sequence identity, which removes homology leakage, gave 0.43 ± 0.02, and the external EXP67S benchmark gave 0.52 and 0.56.

Against ten alternatives the margin is large: the structure-based CNN DeepKa is the strongest competitor at just under 1.0 RMSE, the graph neural network aLCnet is near 1.1, the other ESM2-derived sequence model pKAML is at or above 1.1, and PROPKA3 is comparable to a null model that simply assumes solution pKa values. Critical error rates — residues assigned the wrong protonation state at pH 7 — stay below 1.2% on stratified holdouts and 0.6% on CD-HIT splits, about four times lower than DeepKa. The clear failure mode is engineered buried substitutions in staphylococcal nuclease, which have no evolutionary precedent: RMSE rises to 2.76 for KaML-ESM2 and 1.98 for KaML-ESMC.

#Applications

Ionization states feed directly into enzyme mechanism assignment, molecular dynamics setup, protein engineering of pH-dependent stability, and structure-based drug design where a mispredicted protonation state changes a binding pose. The human proteome scan produced 1,877,704 predicted values, from which 130 cysteines predicted to be deprotonated at physiological pH were mapped to Pfam families; of 30 such cysteines in ubiquitin hydrolases, 29 are annotated catalytic nucleophiles, illustrating use as a functional-site annotation screen. Proteins longer than 1,022 residues fall outside the ESM-2 context limit and were excluded.

#Impact

KaML is the strongest current evidence that residue-level electrostatics can be read out of a protein language model without a structure, and the proteome-wide scan demonstrates a scale no structure-based predictor can match. The practical caveats are concrete: the models are released under CC BY-NC 4.0, so commercial use requires contacting the authors; the ESM-C variant depends on an EvolutionaryScale Forge API token for embeddings rather than local weights; and the work remains a preprint. The command-line interface downloads pretrained weights from Zenodo and can optionally add structure-based KaML-CBT2 predictions, folding an input sequence with ESM-3 when no structure is supplied.