Allo-Allo

The AlloSteric Database annotates allosteric sites in 2,422 proteins. Cross-reference them against the PDB and UniProt so residue numbering agrees, and 653 survive. That number is the whole problem: it is far too small to train a deep model on, and it is why established methods lean on hand-computed pocket geometry from a solved structure instead. Allo-Allo, from Tianze Dong, Christopher Kan, Kapil Devkota and Rohit Singh at Duke University, takes the opposite route — rather than train a predictor on 653 proteins, it uses those 653 proteins to locate allostery inside a model that was already trained on hundreds of millions of sequences.

The mechanism is attention-head selection. Each attention head in ESM-2 produces an n×n matrix of pairwise residue relationships, and some of those heads already fire on the long-range couplings that constitute allostery. Allo-Allo scores every head by the fraction of its total attention mass that lands on known allosteric positions, keeps the heads that survive a Student's t-test with Bonferroni correction, ranks the survivors by signal-to-noise ratio, and row-averages the top K into a per-residue feature vector. A random forest turns that vector into an allosteric probability for each residue. The backbone is never fine-tuned; the forest holds the only learned parameters in the system.

Two properties separate it from the neighbouring work. It labels individual residues from sequence alone, where the PASSer family ranks pockets carved out of a structure, and unlike PLM Allosteric Classification it does not need the orthosteric site as an input — an ablation that restricted attention scoring to orthosteric–allosteric residue pairs performed worse, not better. DeepAllo fine-tunes a language model and fuses it with FPocket geometry; Allo-Allo adds nothing structural at all. The preprint was posted in September 2024 and presented at the Machine Learning in Structural Biology workshop at NeurIPS 2024.

#Key Features

• Structure-free inference: A single amino acid sequence is the only input. No experimental structure, predicted model, or pocket-detection step is required.
• Orthosteric-site agnostic: The method never sees the active site, which removes the annotation dependency that limits comparable residue-level predictors.
• Attention heads beat embeddings: An MLP head over the same frozen 650M embeddings reached 0.68 AUPRC against Allo-Allo's 0.77, indicating the pairwise attention maps carry signal the pooled representation loses.
• Data-efficient by construction: Fifteen scalar features per residue and a random forest extract usable signal from 653 proteins, a regime where a deep prediction head overfits.
• Insensitive to backbone scale: AUPRC moves from 0.73 at ESM-2 8M to 0.75 at 3B, so allostery appears to be encoded even in the smallest checkpoints.

#Technical Details

The deployed configuration pairs frozen ESM-2 650M — 32 layers of 20 attention heads each — with K = 15 selected heads and an attention significance threshold θ = 0.3. The 653-protein set was split 70:10:20; the training and validation halves drove the ablations over K, θ and backbone size, then were recombined to fit a single final classifier that was benchmarked and applied unchanged thereafter. Residues not annotated allosteric were treated as negatives. On the held-out split Allo-Allo reached 0.77 AUPRC, 0.96 AUROC, 0.81 precision, 0.63 recall and 0.71 F1, against 0.46 AUPRC for PASSerRank, 0.30 for PASSer, 0.28 for PASSer2.0 and 0.07 for AllositePro — the 67% AUPRC improvement quoted in the abstract. The selected heads clustered in the later ESM-2 layers, consistent with earlier reports that deeper layers encode higher-order structural relationships.

#Applications

The intended use is early-stage triage of allosteric drug targets, where a modulator binding away from the conserved orthosteric pocket can buy selectivity that a competitive inhibitor cannot. Because it runs on sequence, it scales to proteins with no solved structure — the authors scored 2,567 human cell-surface proteins from the Surfaceome database, a set rich in GPCRs and signal-transduction receptors. Residues in the top percentile of Allo-Allo probability carried significantly higher per-position pathogenicity from AlphaMissense under all three aggregation schemes (Welch's t = 9.46, 9.01 and 8.86; p < 10⁻⁵), and the Allo-Allo score separated the most- from least-risky fifth of positions at 0.59 AUPRC against a 0.44 random baseline.

#Impact

Allo-Allo's contribution is methodological: asking which attention heads of a protein language model respond to a phenomenon can outperform the standard practice of bolting a prediction head onto that model's embeddings, and it does so in the data regime where the difference matters most. The evaluation is retrospective — held-out ASD proteins plus in-silico concordance with AlphaMissense, with no experimental confirmation of a newly predicted site. The work is a workshop paper and bioRxiv preprint that has not been through journal peer review, and the preprint carries no code-availability statement; neither source code nor the fitted random forest has been published, so applying the method means reimplementing the head-selection pipeline from the paper.