MutaPLM

Ask a protein language model what the A205D substitution does to m7GpppX diphosphatase and it returns a likelihood: this residue is more or less plausible than that one under evolutionary pressure. Ask a biochemist and you get a sentence — it enhances decapping activity. The gap between those two answers is the problem MutaPLM sets out to close. Evolutionary plausibility is a single scalar that conflates every reason a residue might be conserved, and it is also the wrong objective for engineering: raising an enzyme's catalytic activity may be costly to a bacterium and therefore selected against, while being exactly what an industrial process needs.

MutaPLM, developed by PharMolix with the Institute for AI Industry Research (AIR) at Tsinghua University and presented at NeurIPS 2024, makes the mutation itself a first-class object in the model rather than an implicit difference between two scored sequences. Its protein delta network takes the subtraction of an ESM-2 representation of the mutant and the wild type, and translates that difference into a feature space shared with natural language. The transformation runs both ways: a delta encoder maps the representation difference into textual semantics so a language model can narrate the effect, and a symmetrical delta decoder turns a written instruction back into a predicted substitution position and amino acid.

Supervision for that alignment comes from MutaDescribe, a corpus the authors built for the purpose: 20.9K wild-type proteins and 171.1K single-site mutations, each paired with a description of its functional effect.

#Key Features

• Explicit mutation representation: The delta network operates on the difference between mutant and wild-type embeddings, so the model represents an alteration rather than two static sequences compared after the fact.
• Mutation explanation as text generation: Instead of a fitness score, the model emits a sentence describing what changes — gain of function, altered substrate specificity, reduced stability — for an arbitrary protein and substitution.
• Instruction-navigated engineering: Candidate mutations are proposed from a free-text objective, letting a user ask for a property that evolutionary plausibility would not favor.
• Chain-of-thought fine-tuning: A two-round dialogue first has the model describe the wild-type protein's function, then conditions the mutational-effect and mutation-prediction steps on that description, transferring function knowledge into mutation reasoning.
• Released training corpus: MutaDescribe is published alongside the checkpoint, with homology-stratified test splits.

#Technical Details

The architecture pairs the 650M-parameter ESM-2 protein encoder with BioMedGPT-LM-7B, a LLaMA2-7B model continually pretrained on biomedical text. Cross-attention modules with 32 trainable query embeddings compress variable-length sequence representations into a fixed set of latent features, and 32 soft tokens carry textual semantics back to the delta decoder. Only the LoRA adapters (rank 16), the wild-type and delta encoders, the delta decoder, the soft tokens, and the position and language-modeling heads are trained — 75.0M parameters, with the remaining 7.4B frozen. Pretraining runs 200K steps on 1.1M protein–text pairs from biomedical publications; fine-tuning runs 70K steps on MutaDescribe, together about 10 days on four A100s. MutaDescribe draws on UniProtKB/SwissProt Phenotypes & Variants annotations enriched from the corresponding PubMed abstracts, with reversed samples added to balance benign against deleterious effects and MMseqs2 homology used to split test proteins into easy, medium, and hard tiers. On explanation, MutaPLM averages 21.34 ROUGE-L against 14.88 for a fine-tuned ESM-2 baseline and 12.42 for GPT-4 assisted by OntoProtein; on engineering it reaches 53.51% accuracy in naming the mutated residue and 40.94 top-50 recall, a 1.6-fold improvement over ESM-2's recall.

#Applications

The explanation task suits variant interpretation and curation work — turning a substitution observed in a clinical or screening context into a hypothesis about mechanism that a biologist can read and test. The engineering task targets directed evolution campaigns where the desired property is easy to write down but hard to encode as a fitness proxy. Beyond single steps, the authors iterate the model with beam search over multiple rounds to accumulate substitutions, evaluating against ESM-based fitness landscapes on AAV, AMIE, avGFP, E4B, LGK, and UBE2I.

#Impact

MutaPLM reframes mutation modeling as a cross-modal problem, and MutaDescribe covers roughly six times as many wild-type proteins as ProteinGym, giving later work a text-annotated benchmark that earlier fitness datasets do not provide. The scope is deliberately narrow: the model handles single-site substitutions only, not multi-point mutants or indels, and performance drops on the low-homology test tier, which the authors attribute to overfitting on out-of-domain proteins. Evaluation is entirely in silico — ROUGE and BLEU against curated annotations, plus a human relevance rating in which 19.4% of predictions were judged accurate — with no wet-lab validation of proposed mutants. Citing misuse risk for pathogenic variant design, the authors state that the model should be restricted to research purposes.