PeptideGPT

A peptide that ruptures red blood cells and one that resists protein adsorption on an implant surface can differ by a handful of residues, and both are rare draws from the distribution a general protein language model learns over natural sequence space. Sampling ProtGPT2 unconditionally and screening afterwards is a poor bargain: on the four tasks studied here, roughly half of what comes out has the desired property. PeptideGPT moves the property out of the filter and into the weights, fine-tuning a separate copy of ProtGPT2 on a corpus containing only sequences that already possess it.

Four such checkpoints were released, one each for hemolytic, non-hemolytic, non-fouling and soluble peptides. They were built by Aayush Shah, Chakradhar Guntuboina and Amir Barati Farimani at Carnegie Mellon University and posted to arXiv in October 2024.

The second half of the method is what the authors call bio-informatic supervision. A causal language model emits amino-acid strings, and a string is not automatically a protein. Generated sequences are ranked by perplexity and the top third kept; each survivor is then tested against convex hulls built from real proteins and discarded if it falls outside; and what remains is folded with ESMFold, keeping only sequences whose predicted structure reaches a pLDDT above 70. Only after those three gates is a candidate scored for the target property.

#Key Features

• One checkpoint per design goal: Hemolytic, non-hemolytic, non-fouling and soluble each get their own fine-tuned model, so a user picks a checkpoint instead of engineering a conditioning prompt.
• Composition-based validity check: Each sequence is embedded by per-amino-acid count, mean position and second central moment, then tested against 20 convex hulls fitted to 571,282 reviewed UniProtKB proteins; between 2.5% and 12% of generations fall outside and are dropped.
• Structure gate before property scoring: Folding every surviving candidate and requiring pLDDT above 70 removes disordered sequences, which is what makes downstream synthesis worth considering.
• Fixed, downloadable weights: All four checkpoints are published on Hugging Face, and the released inference script samples from them directly with no retraining step.
• Base-model comparison on every task: Each fine-tune is measured against unmodified ProtGPT2 under the identical pipeline, isolating the contribution of the property-specific corpus.

#Technical Details

Every checkpoint keeps ProtGPT2's architecture unchanged: a GPT2-large decoder with 36 layers, model dimension 1280 and 20 attention heads of dimension 64, 738 million parameters initialized from ProtGPT2's pretrained weights, a byte-pair-encoded vocabulary of 50,256 tokens, and a 1,024-token context. Fine-tuning is plain causal language modeling on curated single-property corpora: 1,487 hemolytic sequences from DBAASPv3 and Hemolytik, 6,741 non-hemolytic sequences from DBAASPv3 alone, 2,880 non-fouling sequences, and 7,906 soluble sequences derived from PROSO II, with average lengths of 19, 19, 6 and 136 residues. Training ran on a single 48 GB NVIDIA RTX A6000. Sampling uses top-k 950 with a repetition penalty of 1.2, and evaluation covers 5,000 sequences per task across five seeds. Reported figures are generation success rates — the share of structurally valid outputs that a task-specific classifier judges to carry the property — of 76.26% hemolytic, 72.46% non-hemolytic, 78.84% non-fouling and 68.06% soluble, against roughly 45%, 55%, 62% and 52% for base ProtGPT2 through the same pipeline, those four read off a bar chart rather than reported in a table. Scoring uses HAPPENN for hemolysis and PeptideBERT for the other two properties; the classifiers' own accuracies of 70–88% bound how far those numbers can be pushed. Three of the four Hugging Face repositories carry an Apache-2.0 license tag, the hemolytic repository declares Apache-2.0 only inside a nested card so Hugging Face shows no license tag for it, and the code repository has no license file.

#Applications

The checkpoints suit the front of a peptide design funnel, where the goal is many plausible candidates rather than one optimized molecule. Hemolytic generation supplies starting points for membrane-lytic agents; non-hemolytic generation is the safety-side complement for therapeutic peptides; non-fouling sequences are relevant to implant coatings and drug-delivery surfaces that must resist nonspecific adsorption; and the solubility model supplies constructs less likely to aggregate in structural or biochemical work. Because generation runs from a fixed checkpoint on a single GPU, a group can triage thousands of candidates before committing to synthesis. Running the published scoring stage additionally requires cloning the separate PeptideBERT repository and supplying its checkpoint, and it covers only the non-fouling and solubility tasks: no HAPPENN scoring code ships in the repository at all, so the two hemolytic success rates cannot be reproduced from the released pipeline by any route.

#Impact

PeptideGPT is a compact demonstration that a general protein language model can be specialized to a functional property by corpus construction alone, and that pairing it with cheap structural filters raises the fraction of usable outputs enough to matter. The limits are stated plainly by the authors: the evaluation is entirely in silico, with no synthesis or wet-lab assay of any generated peptide, and the success rates measure agreement with a predictor rather than measured activity. Generation is also single-property — designing a peptide that must be simultaneously soluble and non-hemolytic is left to future work.