INFUSSE

Protein language models capture a great deal about a sequence, but some residue-level properties depend on geometry that no sequence context can supply — which residues are packed against which, and how tightly. INFUSSE (Integrated Network Framework Unifying Structure and Sequence Embeddings), from the Department of Mathematics at Imperial College London, is a framework for combining the two systematically and, importantly, for measuring how much each side contributes at every position.

The architecture is deliberately simple. A sequence block takes a residue sequence and its embeddings from a frozen protein language model, passes both through learned transformations, and produces an enriched per-residue representation. A graph block builds a geometric graph over the residues from their spatial coordinates, initializes the node features from the sequence block's output, and runs a diffusive graph convolution over the graph Laplacian. The two outputs are summed. Because the sequence block can be trained and evaluated alone, the difference between it and the full model gives a per-residue measurement of what structure adds.

The demonstration task is B-factor prediction in antibodies — how far each atom fluctuates around its equilibrium position. This is a hard case: antibodies are highly variable in sequence and rich in flexible loops, and general-purpose B-factor predictors degrade sharply when moved onto them.

#Key Features

• Frozen language model backbone: The protein language model is never updated; all learning happens in the transformations and graph parameters above it, so the framework can swap in a different pretrained encoder without retraining it.
• Diffusive graph convolution: The graph block propagates information across the residue graph at a learnable diffusion scale, letting the model choose how far structural context should reach rather than fixing a neighborhood radius.
• Interpretable sequence-structure decomposition: Training the sequence block alone and then adding the graph block yields a per-residue improvement score that localizes where geometry matters.
• Robust to graph construction: Weighted Gaussian graphs, Gaussian Network Model graphs, and a non-diffusive convolution all land within a couple of points of each other, so results do not hinge on one graph recipe.
• Modular by design: The pipeline is packaged so that the language model, the graph construction, and the target residue property can each be substituted.

#Technical Details

The sequence block encodes an input sequence with a frozen ProtBERT model and combines the one-hot sequence and the language model embedding through non-linear transformations. The graph block constructs a residue-level geometric graph — weighted Gaussian by default, with a Gaussian Network Model variant using a 10 Å cutoff — computes its Laplacian, and applies a diffusive graph convolutional network with two weight matrices and a learnable diffusion time. Training is two-stage: the sequence block is fitted first on sequence and embeddings alone, then the graph parameters are optimized jointly with the sequence transformations initialized from that fit. The dataset is 1,510 high-quality PDB structures retrieved via the Structural Antibody Database — 1,143 antibody-antigen complexes and 367 unbound antibodies with B-factor annotation — split ten times into 1,435 training and 75 test entries, with B-factors standardized per entry. Full INFUSSE reaches a mean Pearson correlation of 0.71 ± 0.01 across the ten splits, against 0.64 for the sequence block alone, 0.55 when the language model embeddings are removed but graphs retained, 0.48 for a published bidirectional-LSTM B-factor predictor applied to this dataset, and 0.01 for an unlearned graph Laplacian pseudoinverse baseline. Statistical tests show the graph's contribution is significantly larger in complementarity-determining regions than in framework regions, and larger in loops and less-ordered secondary structure. Code is released as an MIT-licensed Python package.

#Applications

The immediate use is antibody engineering: flexibility at the CDRs and at the antigen interface shapes binding kinetics, developability, and susceptibility to escape mutations, and INFUSSE supplies a per-residue estimate for a designed or crystallized construct without running molecular dynamics. Beyond that specific readout, the sequence-versus-structure decomposition is a general analysis tool: it tells a designer which positions are governed by sequence context a language model already captures and which need the geometry. The framework's modular construction means the same pipeline can be pointed at other single-residue targets.

#Impact

INFUSSE is a compact demonstration that a frozen protein language model plus a lightweight geometric head can beat both sequence-only and structure-only approaches on a residue-level task, and that the improvement is not uniform but concentrated exactly where structural biology would predict: disordered regions, interaction sites, and highly variable positions. That interpretability, rather than the headline correlation, is its main contribution. The scope shown so far is narrow — one property, one antibody dataset — so its generality across other residue properties and protein families remains to be demonstrated.