SpiderGPT

SpiderGPT is a compact protein language model for designing spider silk. Dragline silk owes its combination of steel-like tensile strength and rubber-like extensibility to the repetitive core regions of major ampullate spidroins (MaSps), yet the mapping from repeat sequence to fiber mechanics has resisted description: when the Spider Silkome project sequenced silk genes from 1,098 species and measured fiber properties for 446, individual motif frequencies correlated only weakly with toughness or strength. SpiderGPT attacks that problem generatively, learning the joint distribution over repeat sequences and their measured mechanical properties so that a single checkpoint can be run in either direction.

The model was built by researchers at KTH Royal Institute of Technology, the Swedish University of Agricultural Sciences, and SciLifeLab at Uppsala University, and published in Transactions on Machine Learning Research in 2025 after a first preprint in April of that year. Rather than train a spidroin model from scratch on a dataset far too small to support one, the authors distilled the general-purpose ProtGPT2 into a much smaller student and then specialized it in two low-data fine-tuning stages.

The scope is deliberately narrow. Where earlier generative work on spider silk modeled full-length spidroins, SpiderGPT strips the conserved N- and C-terminal domains and models only the repetitive core, the region the literature identifies as the dominant determinant of fiber mechanics.

#Key Features

• Bidirectional sequence-property modeling: A task token prepended to the input selects between generating a MaSp repeat conditioned on a target property vector and predicting properties from a supplied repeat, so both directions share one set of weights.
• Four-property conditioning: Generation is conditioned on toughness, Young's modulus, tensile strength, and strain at break, each paired with its experimental standard deviation and min-max normalized into an eight-dimensional vector.
• Distillation for a small-data regime: Knowledge distillation compresses a 738M-parameter teacher to roughly 50M parameters, cutting inference time roughly six-fold at comparable perplexity and making the subsequent fine-tuning tractable on a few hundred labeled sequences.
• Parameter-efficient specialization: Both fine-tuning stages use LoRA adapters rather than full-weight updates, which limits overfitting on the 592-sequence labeled set.
• Validated sequence plausibility: Generated repeats are checked against natural MaSps for molecular weight, instability index, isoelectric point, predicted secondary-structure composition, and the distribution of the GGX, poly-Ala, YGQGG, and SV motifs.

#Technical Details

SpiderGPT is a decoder-only transformer with a 512-dimensional embedding, six layers, eight attention heads, a 2048-dimensional feed-forward width, and a 512-token context — roughly 50M parameters against ProtGPT2's 738M, 36 layers, and 1280-dimensional embedding. Stage one distills ProtGPT2 on about 100,000 Araneae protein sequences drawn from UniProtKB. Stage two fine-tunes on 6,000 MaSp repeat sequences from the Spider Silkome dataset under a causal language modeling objective to capture motif structure. Stage three fine-tunes on 592 repeats from 293 species that carry complete experimental mechanical annotations, using five held-out folds of 37 sequences each. On a 185-sequence test set, property prediction reaches a Pearson correlation of 0.8884, Spearman 0.8343, MAE 0.0861, and R² 0.6383, ahead of the SilkomeGPT baseline on every metric (0.8349, 0.7798, 0.0963, 0.5861) despite SilkomeGPT having seen those test instances during its own training. Per-property R² ranges from 0.689 for toughness to 0.793 for tensile strength. BLAST against roughly 11,000 natural spidroin repeats places generated sequences below the 50–60% identity threshold conventionally used to call a sequence novel, while still returning MaSp entries as nearest neighbors.

#Applications

The model is a design tool for engineered silk-like biomaterials: a researcher specifies a target toughness or strain-at-break profile and receives candidate repeat regions to clone into a mini-spidroin construct for expression and artificial spinning, or supplies an existing construct and gets an estimate of the fiber properties it should yield. That supports triage of candidates before wet-lab spinning, which is the expensive step. Downstream targets include biodegradable high-performance textiles and biomedical materials such as sutures, tissue scaffolds, and drug-delivery matrices.

#Impact

SpiderGPT demonstrates that distillation plus staged parameter-efficient fine-tuning can extract a usable sequence-property model from only a few hundred experimentally annotated proteins, a recipe that generalizes to other biomaterial families where mechanical characterization is the bottleneck. Its limitations are real: all validation is computational, with no fibers spun from generated sequences; the labeled set covers 293 species and four properties; and while a LoRA adapter and an inference notebook are public, the base checkpoint the adapter attaches to is not, and no license accompanies either artifact.