TPpred-PepPA

A therapeutic peptide is rarely one thing: the strongest label overlap in this benchmark is between antimicrobial and antiviral peptides (Jaccard coefficient 0.28), and one short sequence can be anticancer, cell-penetrating and toxic at once. Treating those activities as a set rather than as a stack of independent binary classifiers runs into a second problem — the most common label covers 3,852 sequences while the rarest covers 87, an imbalance of roughly 44 to 1. A multi-label head trained with ordinary binary cross-entropy learns the abundant labels and quietly answers "no" to everything else.

TPpred-PepPA splits those two jobs across two training stages. The first learns a shared representation with a transformer encoder-decoder under binary cross-entropy, building one function-aware feature space across all 15 activities. The second freezes that decoder, initializes the encoder from stage one, and fine-tunes 15 function-specific classifiers with Asymmetric Loss, which damps the gradient contribution of abundant negatives so rare positives are not swamped. A decision threshold is then searched per label rather than fixed, so a category with 87 training examples is not made to share a cutoff with one that has 3,852.

The model was built by Bin Liu's group at the Beijing Institute of Technology with collaborators at the Beijing Zhongguancun Academy, Lomonosov Moscow State University, HSE University and Shenzhen MSU-BIT University, and published in BMC Biology. It shares its shape with LAMP-PRo, a compact label-aware classifier over a frozen protein language model, and succeeds the group's own TPpred-LE and TPpred-CMvL.

#Key Features

• Frozen protein language model features: Residue representations come from ProtT5-XL-UniRef50, a 3-billion-parameter protein language model run without gradients and never updated. In the released implementation its 1,024-dimensional per-residue vectors are compressed to 20 dimensions by a fixed, seeded projection that is itself never trained.
• Three-view residue encoding: Every position carries those 20 language-model dimensions, a 20-dimensional one-hot amino acid identity, and a 20-dimensional PSSM column from PSI-BLAST against NR, concatenated into a 60-dimensional input.
• Label embeddings as decoder queries: The decoder consumes 15 learned function embeddings and refines them with function-function attention (how activities co-occur) and function-residue attention (which positions support each activity).
• Asymmetric Loss for the long tail: Weighting positives and negatives differently in the second stage beats Focal, Varifocal and Dice loss under the same schedule.
• Per-label dynamic thresholds: Decision boundaries are searched independently per activity, which is where much of the gain on rare categories comes from.
• Attention that maps to motifs: Without motif supervision, the highest-attention residues fall inside recognized patterns such as AGK, FLP and LKK in antibacterial peptides.

#Technical Details

The benchmark pools PEPred-Suite, SATPdb, DRAMP 2.0 and AntiCP 2.0, drops non-standard residues and sequences of five residues or fewer, and applies CD-HIT at 90% identity, yielding 10,237 non-redundant peptides split 8,190 / 1,023 / 1,024 across training, validation and independent test. Sequences are padded or truncated to 50 residues. The released implementation defaults to a 256-dimensional model, two encoder and two decoder layers, four attention heads, and 30 epochs per stage at a learning rate of 1e-4.

On the independent test set the model reaches 0.561 example-based accuracy and 0.469 label-based F1, against 0.540 / 0.430 for TPpred-CMvL, 0.527 / 0.427 for TPpred-LE, 0.421 / 0.202 for MLBP and 0.403 / 0.359 for PrMFTP. Substituting the backbone costs accuracy: ESM-2 650M gives 0.527 / 0.437 and ESM-3 1.4B gives 0.506 / 0.390, while dropping language-model features entirely gives 0.526 / 0.442. Per-activity binary evaluation is where the separation is widest — AUC 0.923 for anti-inflammatory, 0.918 for antiviral and 0.918 for protein-binding peptides, with MCC on protein-binding peptides rising from 0.153 to 0.627 over the strongest prior method.

#Applications

The intended use is triage: scoring a peptide library for 15 activities at once so that assay capacity goes to the sequences most likely to be multi-functional. The authors host a web server that accepts FASTA input or an uploaded file, which is the only way to use the trained model without retraining it. Running the code locally needs one extra step: PSSM profiles from BLAST against NR, with BLOSUM62 substituted when a profile is missing.

#Impact

TPpred-PepPA is a narrow, single-task predictor rather than a peptide foundation model, and its margins deserve stating plainly: aggregate accuracy is 0.561 against 0.526 for the same architecture with no language-model input, so much of the headline gain comes from the two-stage schedule and thresholding rather than from the backbone. The convincing results are on rare categories, where per-activity MCC moves from near zero to 0.287–0.627. Source code, the benchmark splits and the precomputed PSSM features are published, but no trained checkpoint has been released, and the GitHub copy carries no license at all — the only grant of reuse rights is the identical snapshot archived on Zenodo under CC-BY-4.0, a data license rather than an OSI-approved software license. Using the model outside the web server means retraining it.