KaML-ESMC

The pKa of a titratable side chain determines whether it is charged at physiological pH, and therefore whether it can act as a catalytic acid, a base, a nucleophile, or a salt-bridge partner. Predicting those values has been a structure-based problem for decades: Poisson-Boltzmann solvers, empirical schemes such as PROPKA3, constant-pH molecular dynamics, and machine learning models trained on structures. All of them need a structure, and all are sensitive to which structure you give them.

KaML-ESMC, from Jana Shen's lab at the University of Maryland School of Pharmacy, drops that requirement. It is a small multilayer perceptron task head trained on the frozen per-residue embeddings of ESM Cambrian, predicting the pKa shift of Asp, Glu, His, Cys, Tyr, and Lys residues directly from sequence. The underlying claim is a biological one: because protein language models learn from sequences shaped by hundreds of millions of years of selection, their representations encode not only structure and function but, indirectly, electrostatic character.

A sibling head, KaML-ESM2, was trained identically on ESM-2 embeddings, and comparing the two is the point of releasing both. The ESM-2 variant is slightly more accurate in distribution; this one degrades far less on residues evolution never sampled, and that difference — not the headline RMSE — is what should decide between them for protein engineering work.

#Key Features

• Sequence-only prediction: No experimental or predicted structure is needed, making proteome-scale application feasible for targets that lack structures or whose predicted models contain large low-confidence regions.
• Robustness on designed residues: On engineered buried substitutions in staphylococcal nuclease — the hardest case in the paper, with no evolutionary precedent — error stays substantially lower than the ESM-2-backed head achieves.
• Frozen backbone, thin head: Parameter-efficient fine-tuning of the backbone with qLoRA gave a negligible net change in error, so the released model trains only the task head on frozen embeddings.
• GAINES data augmentation: An augmentation strategy named auGment dAta wIth lateNt spacE Sampling retrieves residues from a protein database whose embeddings closely match a labelled query residue and transfers the experimental pKa label, addressing the scarcity of measured values.
• Separate acid and base channels: Independent heads handle acidic (Asp, Glu, Cys, Tyr) and basic (His, Lys) residues, each deployed as an ensemble of 500 models to give per-prediction standard errors.

#Technical Details

The task head is a four-layer MLP with three hidden layers, taking per-token embeddings from the 6B-parameter ESM Cambrian model and regressing the pKa shift relative to model-peptide reference values. Training used the PKAD-3r experimental database expanded by GAINES, with 50 stratified random holdout splits grouped by unique residue.

Across those holdouts the GAINES-trained model reached an overall RMSE of 0.53 ± 0.02 with a Pearson correlation of 0.97, and the external EXP67S benchmark gave 0.56. Both figures sit just behind the ESM-2-backed head's 0.48 and 0.52. The ordering reverses on the paper's adversarial case: on engineered buried substitutions in staphylococcal nuclease, RMSE is 1.98 against 2.76 for the sibling. Larger, more recent pretraining appears to buy generalization to sequences outside the evolutionary record at a small cost in fit to the ones inside it.

Against ten alternatives both variants lead comfortably: the structure-based CNN DeepKa is the strongest competitor at just under 1.0 RMSE, the graph neural network aLCnet is near 1.1, the other ESM2-derived sequence model pKAML is at or above 1.1, and PROPKA3 is comparable to a null model that simply assumes solution pKa values.

#Applications

Ionization states feed directly into enzyme mechanism assignment, molecular dynamics setup, protein engineering of pH-dependent stability, and structure-based drug design where a mispredicted protonation state changes a binding pose. This variant is the better choice wherever the residues in question are designed rather than natural — pH-dependent stability engineering, de novo enzyme active sites, buried-charge mutagenesis — since that is exactly the regime where its error advantage appears. For bulk annotation of natural proteomes the two variants are close enough that the sibling's local-only inference is usually the more practical option.

#Impact

Releasing two heads that differ only in backbone turns a modelling choice into a measurable one, and the result is more interesting than either number alone: the more accurate model on natural residues is the less reliable one on designed residues. That is a caution worth carrying into any protein engineering campaign that consumes predicted pKa values.

Practical caveats are concrete. This variant depends on an EvolutionaryScale Forge API token for embeddings rather than local weights, so inference requires an external service and an account — a meaningful operational difference from its sibling. The code is released under CC BY-NC 4.0, so commercial use requires contacting the authors, and the work remains a preprint.