ProDVa

Text-guided protein design asks a generative model to turn a description of a desired function into an amino acid sequence. Models such as ProteinDT, Pinal, and Chroma have made the function-alignment side of this problem tractable, but their outputs are frequently not structurally plausible: sequences that a text encoder judges to match the prompt still fold poorly or not at all.

ProDVa, from East China Normal University with collaborators at TeleAI and Fudan University, takes its cue from classical fragment-assembly protein design. The authors first observed that even inserting randomly chosen fragments of natural proteins into generated sequences improves foldability. Building on that, ProDVa augments a protein language model's fixed amino-acid token vocabulary with a dynamic vocabulary: a fragment encoder retrieves natural protein fragments relevant to the input text and adds them as candidate emissions, so the decoder can place a whole functional segment in one step rather than reconstructing it residue by residue.

The result is a model that competes with much larger text-to-protein systems on function alignment while consuming a fraction of their training data, and that produces substantially more well-folded designs. The work was accepted as a NeurIPS 2025 Spotlight.

#Key Features

• Dynamic protein vocabulary: The generation vocabulary is rebuilt per prompt from retrieved natural fragments, so the model emits either a single amino acid token or an entire evidence-backed segment at each step.
• Three-component architecture: A text language model encodes the functional description, a protein language model decodes the sequence, and a fragment encoder embeds retrieved fragments into the decoder's own embedding space via a projection layer.
• Text-conditioned retrieval: Fragment candidates are found by embedding the prompt with PubMedBERT and retrieving the top 16 most similar annotated descriptions from a supporting corpus, using txtai over a Faiss index.
• Extreme data efficiency: Trained on roughly 392K text-protein pairs, ProDVa matches models trained on billions, making the approach viable for low-resource functional domains.
• Two released checkpoints: ProDVa-CAMEO handles function-keyword prompts and ProDVa-Molinst-SwissProtCLAP handles free-form natural-language descriptions; both are published on HuggingFace under an MIT license alongside their training sets.

#Technical Details

The text language model is initialized from GPT-2; both the protein language model and the fragment encoder are initialized from ProtGPT2, giving roughly 1.8B parameters in total. Training optimizes a joint objective combining next-token/next-fragment prediction with a functional-annotation loss, using AdamW at a peak learning rate of 1e-4 with linear warmup over the first 5% of steps and a 1-sqrt decay over the last 10% — 10K steps on the CAMEO subset and 20K on Mol-Instructions, at an effective batch size of 64. Fragment annotations come from InterPro. Training data comprises a 391,933-protein CAMEO-derived subset of SwissProt and 171,089 Mol-Instructions pairs.

On the 507-protein CAMEO test set used by ESM-3, ProDVa reaches 75.88 mean pLDDT with 77.00% of designs above pLDDT 70 and 83.88% below PAE 10, against 66.50 / 47.21% / 33.53% for Pinal and 59.79 / 31.49% / 21.37% for ESM3. Function alignment, measured with ProTrek and keyword recovery, lands within roughly 0.15 points of Pinal despite Pinal training on 1.76B pairs to ProDVa's 392K — under 0.04% of the data. Against ProteinDT and PAAG trained on the same corpus, ProDVa improves pLDDT by 34.35% and PAE by 17.51%.

#Applications

ProDVa suits researchers who can describe a target function in words — an enzyme class, a binding domain, a catalytic motif — but lack a template structure or a labeled dataset to fine-tune on. The keyword checkpoint fits controlled vocabulary workflows built on InterPro or Gene Ontology terms; the Mol-Instructions checkpoint accepts free-form prose. Because the fragment corpus is a swappable retrieval index rather than baked-in weights, the same model can be steered toward a new protein family by changing the supporting documents. It sits alongside InstructPro as an instruction-driven route into de novo design.

#Impact

ProDVa's contribution is a mechanism rather than a scale record: it shows that retrieval of natural sequence fragments is an effective inductive bias for foldability, closing much of the structural-plausibility gap that has limited text-conditioned protein generators. The data-efficiency result is the strongest argument for the approach, since it decouples competitive performance from web-scale text-protein corpora. The authors are explicit about the main limitation: evaluation rests entirely on computational proxies — oracle-model and retrieval-based alignment scores plus predicted structure confidence — and no wet-lab characterization of the designed proteins has been performed.