bio.rodeo
ModelsOrganizationsProvidersLeaderboardAboutSign in
bio.rodeo

The authoritative source for evaluating biological foundation models. No hype, just honest analysis.

Categories
  • DNA & Gene
  • RNA
  • Protein
  • Small molecule
  • Single-cell
  • Spatial omics
  • Pathology
  • Imaging
  • Metabolomics
  • Biosignals
  • Language model
bio.rodeoModelsOrganizationsProvidersLeaderboardAboutFAQSubmit a modelContact
© 2026 Pulsatance. All rights reserved. ~
Built by Pulsatance
Protein foundation models
Protein

C3PI

Western University / McMaster University

Sequence-based protein-protein interaction predictor over ProtT5 embeddings that reaches 0.70 AUROC on the leakage-free gold standard benchmark.

Released: July 2025

Predicting which pairs of proteins physically interact is one of the oldest open problems in computational biology, and for years it looked close to solved: sequence-based deep learning methods routinely reported 95–99% accuracy. In 2024, Bernett, Blumenthal, and List showed those numbers were an artifact of how the benchmarks were split. Existing models were learning almost entirely from sequence similarity between training and test pairs and from node degree — how many partners a protein has in the reference network — rather than from anything resembling binding chemistry. On a human dataset engineered to remove both shortcuts, every method they tested collapsed to roughly random performance.

C3PI (Component Puzzle Protein-Protein Interaction Prediction) is a sequence-based predictor built specifically against that result. Developed by SeyedMohsen Hosseini and Lucian Ilie at Western University with G. Brian Golding at McMaster University, it was posted as a preprint in July 2025 and published in Briefings in Bioinformatics in December 2025. It takes frozen per-residue embeddings from ProtT5, part of the ProtTrans suite, and feeds them through two new components: a puzzler that shuffles blocks of each sequence, and an entangler that mixes the two proteins' features at several length scales rather than only at the end.

It sits in the same family as other task heads over frozen protein language model embeddings, such as PLMDA-PPI and PPLM, but is distinguished by what it is measured against: it is reported as the first sequence-based PPI predictor to improve significantly over random on the leakage-free benchmark.

#Key Features

  • Puzzler augmentation: Each sequence is split into blocks reordered under eight fixed permutations, exposing the network to alternative residue orderings so residues distant in sequence but close in structure can be juxtaposed.
  • Scale-wise entangler: The two proteins' features are fused at each of six resolutions rather than concatenated once at the end, letting the classifier model interactions specific to a given length scale.
  • Multi-scale convolutional encoder: Six parallel 1D convolutional modules span kernel sizes from 795 residues (whole sequence) down to 20, capturing global context and local motifs together.
  • Leakage-free evaluation: Headline results are reported on the Bernett et al. gold standard as well as six species datasets, so they are not inflated by train-test overlap.
  • Ensemble inference: Predictions for the eight permuted versions of a pair are averaged into one interaction probability, the largest single contributor to accuracy in the ablation study.

#Technical Details

Each sequence is padded or truncated to 800 residues and split into 15 blocks of 53. Two weight-sharing branches encode the ProtT5 embeddings through convolutional dense modules with (kernel, stride, output-dim) settings of (795, 1, 16), (400, 200, 32), (200, 100, 64), (100, 50, 128), (50, 25, 256), and (20, 10, 512). At each scale the entangler concatenates the two 2d-dimensional vectors and projects back to d; the six fused vectors are concatenated into a 1008-dimensional representation and passed through a 1008→64→8→1 MLP with ReLU, dropout 0.3, and a sigmoid output. Training used species datasets derived from STRING v11 and filtered at 40% identity with CD-HIT (22,000–55,000 pairs each; 95,864 human training pairs), plus gold standard splits of 163,192 training, 59,260 validation, and 52,048 test pairs.

On the gold standard dataset C3PI reaches 0.703 AUROC, 0.695 AUPRC, 0.665 F1, and 0.293 MCC, against 0.587/0.590/0.372/0.146 for Topsy-Turvy and roughly 0.50 for D-SCRIPT, PIPR, DeepFE, R-FC, and R-LSTM. Across the six species datasets it wins AUROC and AUPRC in all but one case, averaging 13.35% higher AUROC and 26.11% higher AUPRC than Topsy-Turvy. Ablations on the gold standard drop AUROC to 0.646 without the entangler, 0.569 without the puzzler, and 0.549 with neither — essentially random.

#Applications

C3PI is aimed at researchers screening candidate interactions where assays such as co-immunoprecipitation or pull-down are too slow or costly, and particularly at proteins with no close homologs in interaction databases, where similarity-driven predictors have nothing to lean on. The authors demonstrate it on the human NOTCH signalling network, scoring 50 curated binary interactions: C3PI assigns an average score of 0.644 versus 0.490 for Topsy-Turvy, recovering 37 of the 50 above a 50% threshold and 14 above 90%, where Topsy-Turvy recovers none at that confidence.

#Impact

C3PI's contribution is less a large absolute gain than a demonstration that the leakage-free problem is tractable at all — an MCC of 0.293 is well short of usable precision, and the authors say so plainly. Its value is in establishing a non-random baseline on a benchmark where everything else sat at chance, and in showing through ablation that a simple augmentation carries most of that gain. Practical adoption is constrained: the repository ships training, embedding, prediction, and evaluation scripts alongside an externally hosted ~800 MB trained checkpoint, but carries no license file, and the published article is available under CC BY-NC 4.0.

Citations

DOI: 10.1093/bib/bbaf685

Preprint

DOI: 10.1101/2025.07.26.666948

Recent citations

Papers that recently cited this model.

Not enough citation data yet.

Top citations

The most-cited papers that cite this model.

Not enough citation data yet.

Where to run C3PI

Providers that host C3PI for inference, fine-tuning, or weight download.

No providers recorded yet. Browse all providers

Fields of citing research

Not enough data

Openness

bio.rodeo opennessClosed · low usability and reproducibility
21Closed
Usability — can I run it?19
Reproducibility — can I retrain it?26

Tags

cnninteraction_network_inferenceprotein_protein_interaction_predictionproteomicsrepresentation_learningsupervised

Resources

GitHub RepositoryResearch PaperbioRxiv Preprint