ProtBLIP2-SST

Protein function prediction has long been framed as classification: assign a protein to a set of Gene Ontology terms, or predict labels with a multi-label classifier. That framing is convenient for benchmarking but rigid in practice, because a fixed label set cannot simultaneously express molecular function, biological process, subcellular location, and the free-text narrative a curator would actually write. Generation-based annotation instead reframes the task as captioning — given a protein, produce a sentence describing what it does.

ProtBLIP2-SST, from Zhuoyang Chen and Qiong Luo at the Hong Kong University of Science and Technology, adds the modality those systems omit: three-dimensional structure. Prior protein–text models align sequence embeddings with text, either contrastively as in ProtST or through a querying transformer feeding a language model. ProtBLIP2-SST keeps that architecture but swaps the sequence-only encoder for SaProt, a protein language model whose vocabulary fuses residue tokens with Foldseek-derived 3Di structural tokens, so structure enters the pipeline without a second encoder or an extra alignment stage.

The result is the BLIP-2 recipe from vision–language modeling transplanted to proteins: a frozen protein encoder in place of the frozen image encoder, a Q-Former as the cross-modal projector, and a frozen scientific LLM as the decoder. It was posted as a bioRxiv preprint in July 2026 under a CC BY license.

#Key Features

• Structure-aware encoding without a second encoder: SaProt's structure-aware vocabulary carries sequence and AlphaFold-predicted geometry in one token stream, avoiding the separate structural branch that co-encoder designs require.
• Q-Former cross-modal projector: Eight learnable query tokens cross-attend to the frozen encoder's residue features and are trained against a frozen BiomedBERT text encoder, compressing a variable-length protein into a fixed set of language-aligned vectors.
• Three alignment objectives: Stage 1 trains protein–text contrastive learning, protein–text matching with hard-negative mining, and protein captioning jointly; ablations show the captioning objective contributes the most.
• LoRA-adapted decoder: Stage 2 linearly projects the aligned queries, prepends them to prompt embeddings, and fine-tunes the LLM with low-rank adapters rather than full fine-tuning, preserving its pretrained linguistic ability.
• Field-stratified analysis: Performance is broken out by caption field, isolating where structural information helps rather than reporting only a single aggregate score.

#Technical Details

Stage 1 aligns a frozen SaProt encoder (evaluated at both the 35M and 650M SaProt_*_AF2 checkpoints) with a frozen BiomedBERT text encoder through a Q-Former using eight query tokens, trained for 50 epochs with AdamW, learning rate 1e-4, and weight decay 0.05 on two NVIDIA A100 80GB GPUs. Stage 2 connects the warm-started Q-Former to a frozen Galactica-1.3B decoder and trains for 10 epochs with LoRA (rank 8, alpha 16, dropout 0.1) applied to the self-attention query, key, value, and output projections. Sequences are truncated to 1024 residues and captions to 128 tokens.

Training data comes from the Swiss-Prot V3 protein–text corpus — roughly 441,000 pairs — intersected with AlphaFold Database v4 predicted structures, retaining only records with an available prediction, for a final split of 410,925 training, 9,236 validation, and 9,239 test proteins. On stage 1 retrieval, ProtBLIP2-SST improves test-set accuracy and Recall@20 over sequence-only Q-Former baselines at matched parameter scales, with the 650M variant scoring highest. On stage 2 captioning, the 650M model leads on BLEU, ROUGE, and METEOR and places second on exact match; the 35M model gains +0.63 BLEU-4, +0.97 ROUGE-L, and +0.63 METEOR over its sequence-only counterpart at comparable exact match (30.59 vs. 30.33). Ablations against ESM-2 and ProstT5 encoders confirm the structural contribution, and LoRA fine-tuning Galactica directly on raw amino-acid strings reaches only 24.4% exact match.

#Applications

The model targets annotation of proteins that lack curated descriptions — metagenomic open reading frames, hypothetical proteins, and newly sequenced isolates — where a free-text caption naming a probable molecular function, subcellular location, and homology context is more useful to a biologist than a GO term list. Because stage 1 is trained contrastively, the aligned encoder also supports protein-to-text and text-to-protein retrieval, letting researchers search a proteome with a natural-language description of the activity they want.

#Impact

ProtBLIP2-SST is a preprint and has not yet been peer reviewed; no code or weights have been released, and coverage is limited to proteins with an available AlphaFold Database prediction. Its contribution is a controlled demonstration that structure-aware tokens improve generative protein annotation under an otherwise identical BLIP-2 pipeline, and that the gain is concentrated in mechanistic FUNCTION descriptions rather than homology-driven similarity labels. The authors also document a decoder bias — Galactica tends to open captions with FUNCTION regardless of the ground-truth field — pointing toward field-balanced decoding as the next step for this class of model.