GLProtein

Protein language models trained by masked modeling over sequence alone leave structural information on the table, and the models that add structure back usually add one kind of it. Approaches like SaProt encode local geometry as a structural alphabet appended to each residue; graph encoders such as GearNet build a representation from the 3D neighborhood of each amino acid. Both operate inside a single protein.

GLProtein, from the Hong Kong Polytechnic University and Sichuan University, argues that structural signal lives at two scales that a pretraining objective should capture simultaneously. Globally, proteins relate to one another through fold similarity — a relationship that exists between proteins, not within them. Locally, each amino acid is itself a molecule with a chemical substructure that residue-index embeddings discard entirely. GLProtein combines four training signals accordingly: masked protein modeling, triplet scoring over protein-protein structural similarity, 3D distance encoding, and substructure-based amino acid encoding. The authors present it as the first protein pretraining framework to incorporate both scales.

The paper was accepted to EMNLP 2025 Findings, placing it in the same family as ProtTrans and ProtST — general-purpose pretrained encoders evaluated by transferring to many downstream prediction tasks rather than by a single headline capability.

#Key Features

• Triplet structure-similarity scoring: TM-Vec supplies structural-alignment scores over sampled protein pairs, which become positive and negative examples in a triplet objective that teaches the encoder how proteins relate by fold.
• Substructure-based amino acid encoding: Each residue is represented through a mol2vec embedding of its molecular substructure, giving the model chemical detail that a 20-way token vocabulary cannot express.
• 3D distance encoding: Interatomic distances from AlphaFold DB structures are folded into attention as a distance bias, injecting geometry without a separate graph encoder.
• Structure-free at inference: All downstream tasks are fine-tuned from sequence alone, so the structural supervision used in pretraining imposes no requirement for a predicted or experimental structure at deployment.
• Both fine-tuned and frozen use: The released code supports task-specific fine-tuning and frozen embedding extraction, the latter used for semantic-similarity and binding-affinity evaluation.

#Technical Details

The encoder is initialized from a pretrained ProtBert BERT-style transformer and pretrained on Swiss-Prot — roughly 600,000 manually curated sequences — with 3D coordinates drawn from AlphaFold DB. Training runs 300,000 steps at a learning rate of 1e-5 with weight decay 0.01 across four NVIDIA A6000 GPUs. Within each block, the distance-aware attention output is added to the residue representation and passed through a residual MLP; the overall objective sums the masked-modeling loss with the structure-similarity and substructure terms.

Evaluation spans amino acid contact prediction, secondary structure, remote homology detection, stability, fluorescence, protein-protein interaction identification, binding affinity, and semantic similarity inference. On long-range contact prediction over the ProteinNet and TAPE benchmarks, GLProtein reaches P@L of 79.2 against 71.2 for KeAP and 68.3 for ESM-2, with consistent gains at short and medium range as well. On protein-protein interaction identification it leads on SHS27k, SHS148k, and STRING under both breadth-first and depth-first splits, ahead of SaProt, ESM-C, GearNet, and MAPE-PPI, though the margin narrows as the fine-tuning set grows and pretraining matters less. The authors report two tasks where it does not win: ESM-2 is stronger at predicting binding affinity changes from mutations, which they attribute to GLProtein's lack of mutational signal, and the fluorescence task — which requires discriminating near-identical variants — shows no meaningful improvement.

#Applications

GLProtein serves as a drop-in sequence encoder for groups building predictors of protein function, interaction, or biophysical properties. Because structural supervision is confined to pretraining, it can be applied to sequences with no known or predicted structure, which matters for metagenomic and designed proteins. The protein-protein interaction results make it a reasonable default for interactome modeling, while the reported weaknesses suggest looking elsewhere for mutational-effect tasks.

#Impact

The concrete finding is that structure-similarity supervision between proteins, not just geometry within them, measurably improves a sequence encoder — most visibly on long-range contacts and interaction prediction, the tasks where fold-level relationships matter. The model is a research artifact rather than a production system: the code is MIT-licensed and documents both fine-tuning and embedding-extraction workflows, but it depends on TM-Vec and mol2vec as external components, and the pretraining corpus of roughly 600,000 Swiss-Prot sequences is small next to the hundreds of millions used by contemporary protein language models, which bounds how far the comparison to those models can be pushed.