RegimeFormer

The same amino-acid substitution can be catastrophic in one protein and unremarkable in another. Deep mutational scanning measures that difference directly, but each experiment is tied to one protein, one readout and one laboratory. Protein language models fill the gap by scoring substitutions from local sequence context — yet they carry no explicit notion of the protein background a substitution lands in.

RegimeFormer supplies that missing variable. It fits a continuous protein-level regime coordinate, a single number placing a whole protein on an axis running from fragile (most substitutions are damaging) through adaptive (many are tolerated or beneficial), and then conditions substitution-effect prediction on it. The coordinate is deliberately not a proxy for taxonomy or family: adjusted mutual information between regime label and taxonomy is 0.0063, and language-model features explain 44.3% of coordinate variance against 1.0% for taxonomy, length and Pfam annotations combined.

The sequence encoder is somebody else's. Proteins are embedded with a pretrained ESM-family model — the production residue scan uses a frozen ESM-2 t33 650M UR50D backbone — and the regime signal transfers across backbones, with mean cross-backbone correlation of 0.700 between ESM-2, ESM-1v and ProtT5 (of the ProtTrans suite). New here is the regime coordinate itself and the regime-conditioned substitution head trained on top of the frozen representations. The work comes from a 15-author collaboration led by Nanyang Technological University, with Dacheng Tao and Siyuan Ma corresponding, posted to arXiv in August 2026 and to bioRxiv days later.

#Key Features

• Regime-conditioned substitution head: For each residue and candidate mutant, the model combines the residue-level language-model representation, learned wild-type and mutant amino-acid embeddings, the protein regime vector, an assay-context embedding and optional structural features, passing them through six regime-aware attention blocks with FiLM-style assay conditioning.
• Three output heads: Continuous effect size, predictive standard deviation, and damaging/tolerated/beneficial class probabilities. The uncertainty head cuts expected calibration error from 0.094 to 0.024 and lifts selective Spearman from 0.548 to 0.771 at 20% coverage.
• RegimeAtlas: A harmonized index of 202,556,313 non-redundant sequences, distilled from 424,786,370 raw records drawn from UniProtKB, RefSeq, Ensembl/GENCODE, MGnify and metagenomic collections, with a regime coordinate computed for every entry.
• Precomputed residue scan: 995,995 proteins and 407,048,356 residues carry stored fragility, adaptability and uncertainty tracks, with full L×19 substitution matrices served on demand — 0.35% of the storage a full matrix dump would need.
• Predictor-difficulty forecasting: Regime position ranks where independent variant-effect predictors incur large errors, reaching Spearman 0.533 with observed error and AUROC 0.861 on high-error cases.

#Technical Details

Training used a diversity-preserving one-million-protein subset of the atlas, with the whole campaign — atlas construction, indexing, optimization and the residue scan — running about five months on a 32-GPU NVIDIA A100 cluster at a 1,022-residue window. On 217 substitution assays from ProteinGym and MaveDB, RegimeFormer reaches aggregate Spearman 0.698 against 0.662 for Kermut and 0.613 for ProteinNPT. Ablations isolate each input: the full model reaches substitution-level Spearman 0.723, a language-model-only variant 0.579, and removing the regime coordinate alone gives 0.701. Under strict shift the model retains roughly 82%, 62% and 38% of in-distribution substitution performance on unseen-protein, unseen-family and low-homology (<30% identity) splits, 92.3% under species hold-out, and a frozen post-cutoff temporal challenge moves mean Spearman by only −0.006.

#Applications

The atlas is built to be spent as a screening budget. In retrospective replay at a 5% candidate budget, ranking by RegimeFormer recovered beneficial variants at HitRate 0.540 versus 0.434 for the strongest baseline; in a blinded study, experts given regime coordinates, virtual DMS maps and uncertainty reached HitRate@20 of 0.41 against 0.29 working from sequence and annotations alone. High-fragility residues are strongly enriched at active sites (odds ratio 11.3), ligand-binding sites and disease-associated positions, which makes the residue tracks usable for nominating saturation-mutagenesis windows in proteins with no measured landscape. Regime-derived priors also feed transcriptomic perturbation and PRISM drug-response models, and the released code ships a locally run RegimeAtlas Explorer for browsing the atlas, residue tracks and substitution matrices.

#Impact

RegimeFormer's contribution is conceptual as much as numerical: it argues that perturbational behaviour is an organizing axis of protein space alongside sequence, structure and family, and that a protein-level prior helps most where local sequence neighbourhoods are sparse — the low-homology regime where variant-effect prediction is hardest. The ablations keep that claim modest, since the regime coordinate is the smallest of the model's learned contributions. Practical limits are stated plainly by the authors: atlas-scale inference success falls to 83.4% for sequences longer than 10,000 residues, and the drug-response bridge collapses on unseen compounds (R² = 0.038). The work is a preprint awaiting peer review; the source code is distributed as a single archive under an academic, non-commercial license, and trained weights are available from the corresponding authors on request rather than by download.