IndeLLM

Protein language models score missense variants by comparing the likelihood assigned to the reference amino acid against the likelihood assigned to the substitution — same position, same length, a clean comparison. In-frame insertions and deletions break that setup. The mutant protein differs in length from the wild type, so the two pseudo-log-likelihoods being subtracted are sums over different numbers of terms, and the difference grows with indel size regardless of whether the indel is damaging. Indels account for roughly 18% of human genetic variation and are far less studied than single-nucleotide variants, leaving both a methodological gap and a shortage of labelled data.

IndeLLM, from Oriol Gracia i Carmona, Vilde Leipart, Christine Orengo and Franca Fraternali at University College London, closes the mismatch with a deliberately simple move: compute the pseudo-log-likelihood only over the amino acids present in both sequences. Because a language model is context-aware, an indel perturbs the predicted marginals of the residues around it, so its effect remains visible in the shared positions — but now the two sums have the same number of terms and are directly comparable. Nothing is trained; the underlying model is frozen and used purely for inference.

The same frozen embeddings then support a supervised layer. A small Siamese network trained on indel labels turns the zero-shot signal into a stronger classifier, and a per-residue readout of the probability shift can be painted onto a protein structure to show which regions the model believes the indel destabilises.

#Key Features

• Overlap-restricted likelihood scoring: Comparing pseudo-log-likelihoods over the shared residues alone removes the length-dependent bias that afflicts whole-sequence scoring of indels.
• No fine-tuning required: The zero-shot score needs one forward pass per sequence with frozen weights, avoiding the overfitting risk of training on small pathogenicity datasets.
• Species- and length-agnostic: The underlying language model saw protein sequences from across the tree of life, so scoring is not restricted to human transcripts, and the method handles indels from one residue to hundreds.
• Interpretable per-position readout: Subtracting wild-type from mutant per-residue probabilities yields a value between -1 and 1 at every position, which the notebook maps onto the 3D structure to localise the predicted damage.
• Optional supervised head: A one-hidden-layer Siamese network over the frozen embeddings, plus indel type and length, converts the score into a calibrated pathogenic/benign classifier.

#Technical Details

Embeddings and likelihoods come from ESM-2 at 650M parameters, selected after benchmarking eight protein language models including ProtBert, ESM-1v, ESM-1b, and ESM-2 checkpoints from 35M to 15B parameters. Sequences over 1,000 residues are cropped to a 500-residue window either side of the indel to stay inside the attention span. The evaluation set merges three published indel collections into 7,500 unique in-frame indels — 2,409 insertions and 5,091 deletions, 2,878 pathogenic and 4,622 benign, spanning 1 to 223 residues.

Scoring over the overlap raises the Matthews correlation coefficient from 0.39 to 0.65 and the AUC from 0.73 to 0.86 relative to whole-sequence scoring on the full dataset; masking each position individually rather than reading logits from a single encoding pass gives no benefit at far higher cost. The Siamese network splits the data 80/10/10 after clustering wild-type sequences at 50% identity with CD-HIT to prevent leakage. Of four architectures tested, the one separating embeddings of the overlapping region from embeddings of the inserted or deleted residues performs best, reaching MCC 0.77 on the test split against 0.73 for PROVEAN, 0.62 for the IndeLLM score itself and 0.57 for MutPred-Indel — mostly through a drop in false negatives among insertions, from 19.5% to 6.1%.

#Applications

The immediate users are clinical genetics and variant-curation teams facing in-frame indels of uncertain significance in genes such as FGFR1 and GLMN, who currently rely on tools built from hand-engineered features and tuned to human data. IndeLLM runs from a Colab notebook with no local setup, returns both a call and a per-position damage profile that can be inspected against known functional sites, and extends to non-human proteins — relevant to agricultural and model-organism genetics, where indel annotations are scarce.

#Impact

The contribution is narrow and useful: a correction to how likelihood-based scores are computed for length-changing variants, recovering most of the performance gap between zero-shot language model inference and purpose-built indel predictors at a fraction of the compute. It also sharpens what transfer learning adds — the supervised head's advantage traces specifically to insertions whose added residues look plausible in context, a failure mode the likelihood score cannot see. The work is published in Patterns and distributed as MIT-licensed code with the full dataset, splits and analysis notebooks. Its scope is bounded: in-frame indels only, effects predicted from sequence with no experimental validation, and a benchmark drawn from the same ClinVar, gnomAD and DDD sources labelling every competing method.