Loading…
Joint sequence and backbone co-design from a single-stage transformer trained directly on 3D coordinates, with no structure tokenizer.
No providers recorded yet. Browse all providers
A protein backbone is already a list of 3D coordinates, but most recent sequence–structure co-design models cannot generate one directly. They first train an autoencoder to compress backbones into a discrete structure alphabet, then train a second generative model over that alphabet alongside amino acid tokens — two stages, plus a learned vocabulary that bounds what the generator can express. SimpleDesign, from Apple's machine learning research group, removes the first stage: coordinates stay continuous, and one model is trained end to end in a single run.
Training corrupts both modalities of the same protein at once and asks the model to restore them. Amino acids are randomly masked and recovered under cross-entropy; Cα coordinates are interpolated toward Gaussian noise and recovered by regressing a velocity field under mean-squared error. The two corruption levels come from independent timesteps, t for sequence and t' for structure, so one run covers the whole unit square between them: clean sequence with noised structure is folding, clean structure with masked sequence is inverse folding, and the interior is co-design.
That places the model between two families. Tokenized multimodal protein language models such as ESM-3 and DPLM-2 get scalable masked modeling but pay for a structure tokenizer; geometric models such as Proteina, La-Proteina and RFdiffusion define specialized SE(3)-aware processes over residue frames. SimpleDesign takes neither route, and follows SimpleFold in the same group's minimalist line.
Training used the filtered AFESM corpus, an integration of the AlphaFold Database and the ESM Metagenomic Atlas covering roughly 800M predicted structures clustered into about 5M non-singleton clusters. Keeping only cluster representatives of length 32–512 residues with pLDDT above 85 leaves 1,807,333 structures, 1,000 held out for validation; unusually for protein design, no secondary-structure filter is applied, so coil-heavy and disordered regions stay in. The model trains for 300,000 steps on 64 NVIDIA H100 80GB GPUs at an outer batch size of 128, then for 50,000 more on 442,511 AFDB SwissProt entries filtered the same way. Sinusoidal encodings and RoPE over a shared residue index align the modalities without cross-attention, and rigid-body augmentation with Kabsch alignment of loss targets supplies rotation invariance in place of equivariant layers. No parameter count is reported.
On unconditional co-generation (100 samples each at lengths 100 through 500), SimpleDesign at γ = 0.3 reaches 0.53 co-designability under scRMSD ≤ 2 Å and 0.74 under scTM ≥ 0.9, against 0.30 / 0.46 for DPLM-2 and 0.09 / 0.11 for ESM-3, at 0.97 novelty against the PDB. Generated sequences score 5.18 ± 4.13 perplexity under ProGen2 and 81.19 ± 12.27 mean ESMFold pLDDT, and structure-only designability from inverse-folding with ProteinMPNN and refolding reaches 0.44 / 0.63. The ablation is the sharpest result: a vanilla transformer with fully shared parameters matches or beats the Mixture-of-Transformer variant, at 0.62 / 0.84, which the authors read as evidence that the objective rather than the architecture is doing the work.
SimpleDesign generates de novo backbones together with sequences intended to fold into them — the entry point for scaffold and binder pipelines that today chain a structure generator to a separate inverse-folding model. Its objective also covers the folding and inverse-folding edges of the timestep square, though the reported experiments are confined to unconditional generation. The audience is methods researchers rather than design teams: outputs are Cα traces without side chains, evaluation spans lengths 100–500, and nothing was tested at the bench.
The contribution is a negative result stated constructively: structure tokenization is not a prerequisite for competitive co-design. The authors claim no superiority over specialized geometric models — MultiFlow still leads several co-designability columns, and SimpleDesign's FoldSeek cluster diversity trails DPLM-2, which the paper attributes to training-data curation rather than the objective. What it establishes is a baseline that scales with data instead of pipeline complexity, and that inherits improvements from general transformer research rather than protein-specific architecture work. It appeared in Transactions on Machine Learning Research in August 2026 with no code or weight release, so reproducing it means reimplementing from the description.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.