VenusVaccine

Reverse vaccinology screens a whole pathogen proteome for the few proteins that might raise protective immunity, and the tools that do it have almost all compressed each antigen into a fixed-length vector first. VaxiJen and its successors fold per-residue physicochemical descriptors together through an auto-cross-covariance transform, so a 200-residue protein and a 900-residue one reach the classifier as the same short list of numbers, and a solvent-exposed loop counts for no more than a buried core. That compression caps their accuracy, and it is why a classifier fitted on bacterial antigens transfers badly to viruses.

VenusVaccine keeps per-residue resolution and lets attention decide which residues carry the signal. A protein is encoded three ways at once: amino-acid embeddings from a frozen protein language model, and two structural tokenizations of the same backbone — Foldseek 3Di tokens, summarizing the geometry between neighbouring residues in a 20-symbol alphabet, and ESM-3 structure tokens, whose 4,096-entry vocabulary describes coarser local substructures. Two cross-attention modules run in parallel, one per scale, each drawing queries from the structure tokens and keys and values from the sequence embedding, with rotary position embeddings supplying relative position. The sequence stream is asked, in effect, what each piece of local geometry should attend to.

The model came out of Shanghai Jiao Tong University, East China University of Science and Technology and Matwings Technology, and was published at ICLR 2025. It belongs to the same AI4Protein family as VenusPLM, and approaches the immunogenicity question from the opposite end to ImmunoStruct — whole antigens rather than peptide-MHC complexes.

#Key Features

• Dual attention over two structural scales: Foldseek tokens capture atomic-level contacts between adjacent residues while ESM3 tokens describe local folds, and the two streams use both without committing to a single resolution.
• Attention pooling that reads as an epitope map: a masked 1D convolution produces one weight per residue before pooling, and those weights are interpretable — on the SARS-CoV-2 spike they concentrate at positions 150–164, 291–320 and 439–478, overlapping the receptor-binding domain.
• Hand-crafted descriptors as an auxiliary channel: five E-descriptors and three Z-descriptors per residue join the pooled representation alongside the learned features rather than replacing them.
• Three domain-specific checkpoints: Bacteria.pt runs on a frozen ProtBert backbone, Virus.pt and Tumor.pt on frozen Ankh-large; all three ship fixed, so inference needs no training step.
• ImmunoDB: a curated benchmark of 7,216 labeled antigens spanning bacteria, viruses and tumors, released with train, validation and test splits.

#Technical Details

Only the adapter is trained; the backbone is loaded frozen and runs under torch.no_grad(). Each cross-attention module uses eight heads, and the pooled representation concatenates the sequence embedding, both attended structural streams and the E/Z descriptors before a two-layer classification head. Training used AdamW at a learning rate of 5e-4, weight decay 0.01, dropout 0.1, a 4,000-token batch cap and early stopping with patience 5, on a single RTX 3090. ImmunoDB was assembled from Protegen, IEDB, UniProt, NCBI and the primary literature through October 2023, with negatives drawn from pathogen proteomes at under 30% identity to any positive and capped at 25–1,024 residues: 913 positive and 1,562 negative for bacteria, 2,078 and 1,886 for viruses, 300 and 477 for tumors. Averaged over ten random splits, the best configuration reaches 84.5% accuracy and 65.9 MCC on bacteria, 92.2% and 84.3 on viruses, and 76.9% and 55.0 on tumors, against 83.3% and 63.2 for VaxiJen 3.0 on bacteria. An ablation on input structures found AlphaFold 2 backbones more accurate downstream than ESMFold ones.

#Applications

The intended use is narrowing a proteome to a candidate list before anyone orders peptides. On the Helicobacter pylori proteome — 1,858 proteins after redundancy removal — it flagged 123 candidates containing all 11 experimentally confirmed protective antigens, a fold-enrichment of 15.1 against 8.8 for the best tree-based baseline. On 38 SARS-CoV-2 proteome sequences it ranked the spike glycoprotein first, the target of five of the nine WHO-approved vaccines. Running it means supplying a structure, converting it to per-protein features with Foldseek and the ESM3 structure encoder, then calling infer.py with a pathogen type.

#Impact

The contribution that will outlast the leaderboard is ImmunoDB: immunogenicity models had been fitted on a few hundred samples per pathogen class, and a three-class benchmark makes cross-species generalization measurable rather than assumed. The attention-weight readout is the second durable piece, giving a residue-level rationale checkable against known epitopes. Two limits are worth stating plainly. The code carries a CC-BY-NC-ND-4.0 license, ruling out commercial use and derivative works, and the three checkpoints — distributed from a public Google Drive folder with no license of their own — inherit those terms; only the ImmunoDB datasets are permissively licensed, under Apache-2.0. And this is a supervised adapter over a frozen backbone rather than a model with a pretraining corpus of its own, so its ceiling is set by those labels, and the case studies are retrospective checks rather than new wet-lab work.