ESMCapsid

When a profile HMM searches an environmental metagenome for capsid proteins, it is really asking whether a new sequence resembles capsids somebody has already collected. Once a phage's major capsid protein has drifted below roughly 30% identity to every reference, that search returns nothing — not because the protein has stopped being a capsid, but because the evidence the search reads has run out. The HK97 fold it adopts is unchanged, and so are the packing constraints imposed on it by having to close into a shell. ESMCapsid is built on the premise that a protein language model's representation still carries that architectural signal after primary-sequence similarity has become undetectable.

ESMCapsid was developed by Sihang Liu, Siqing Xia and Hong Wang at Tongji University and the Shanghai Institute of Pollution Control and Ecological Security, and posted to bioRxiv in September 2026. It is released as two checkpoints derived from the same ESMC 600M base: ESMCapsid-S, a LoRA fine-tune trained with a supervised contrastive objective for high-throughput capsid screening, and ESMCapsid-C, a full continued-pretraining run on capsid sequences that supplies the representation used for everything downstream.

The split exists because the authors' benchmark showed those two demands are not the same. Four general-purpose protein language models — ESM-2 650M, ESM-3 open, ESMC 600M and Profluent's E1 600M — all separated capsid from non-capsid proteins at F1 above 0.94 under random splits, yet every one degraded on family-held-out and low-identity splits and showed higher masked-language-modelling loss on capsids than on other proteins. Accurate binary discrimination did not imply a faithful model of within-capsid diversity.

#Key Features

• Two-stage screening at layer 16: a primary classifier on mean-pooled layer-16 ESMCapsid-S representations applies a 0.99 capsid-probability threshold, and survivors are re-scored by a second classifier trained on hard negatives, holding F1 above 0.992 at a 0.008% false-positive rate while halving inference cost against full-depth inference.
• Semantic motif tokens: a sparse autoencoder over layer-28 ESMCapsid-C residue embeddings, followed by Leiden clustering of token activations, yields 364 motif tokens that turn each protein into an ordered motif string comparable across proteins with no detectable homology.
• Architecture-aware embedding space: HDBSCAN over ESMCapsid-C sequence embeddings resolved 99 capsid groups, 97 of which map onto nine capsid architecture categories, with mean dominant-taxon purity of 97% at realm and kingdom level.
• Geometry recovered from sequence alone: conserved motif n-gram cores projected onto resolved HK97-like and picorna-like shells concentrate in recurrent protomer bands and a shared mid-shell band, at restricted radial positions rather than diffusely.

#Technical Details

Both checkpoints are 36-layer transformers with hidden size 1,152 and 18 attention heads, initialised from the Synthyra ESM++ Large port of ESMC 600M. ESMCapsid-S was trained for 10 epochs with AdamW at a learning rate of 1e-5; ESMCapsid-C updated all parameters for 100,000 steps of 15% masked-language modelling on a redundancy-filtered corpus of more than 210,000 capsid proteins drawn from UniProt, NCBI Virus, IMG/VR and MGnify Proteins. The binary benchmark held 5,212 capsid proteins against 40,000 viral non-capsid and 150,000 cellular proteins, split three ways: random, 30%-identity-held-out and family-held-out. Applied to 343,633,104 AAI50 clusters derived from over 830 million MGnify and IMG/VR proteins, the screen returned 216,296 putative capsid clusters covering 2,701,690 sequences — 49-fold more than NCBI Virus's capsid collection — of which profile-HMM search recovered only 38.12%. AlphaFold 3 predictions for HMM-unsupported groups gave within-group TM-scores averaging 0.560 against 0.326 between groups of the same architecture.

#Applications

The immediate use is viromics annotation: running ESMCapsid-S over a protein catalogue surfaces capsid candidates that homology search cannot reach, and the ESMCapsid-C embedding then places each candidate in an architecture category rather than leaving it as an unannotated hit. Because motif profiles are comparable across proteins with no shared ancestry, the same pipeline supports habitat-level questions — the study found HMM coverage varying from 43.0% in plant-associated habitats to 73.3% in animal-associated samples — and flags groups such as cluster 88, internally coherent but matching no known capsid or non-capsid structure, as targets for structural work.

#Impact

ESMCapsid demonstrates that sparse autoencoder interpretability, so far mostly applied to general protein language models, can be pushed into a specific structural biology question and produce features that correspond to positions in an assembled particle. Its practical claim — that most environmental capsid diversity is invisible to reference profile HMMs — rests on computational evidence alone, and the authors are explicit that candidate novel architectures remain hypotheses until experimentally validated. One-dimensional sequence representations also handle BTV-like capsids poorly, where repeat register and helix packing carry the architecture. The work is a preprint awaiting peer review, and the released weights inherit EvolutionaryScale's non-commercial license from their ESMC base.