BetaDescribe

Protein2Text was the preprint's title; BetaDescribe is the system it introduces, and the name every released artifact carries. Built by Edo Dotan and colleagues at the Technion and Tel Aviv University, posted to bioRxiv in December 2024 and published in PNAS in June 2026, it takes an amino acid sequence and writes a paragraph describing the protein: its molecular function, catalytic activity, metabolic pathway involvement, subcellular localization, domain content, and post-translational modifications.

The problem it targets is the long tail of unannotated proteins. Functional annotation in practice means transferring a description from the closest characterized homolog, which works well when one exists and fails silently when none does. General-purpose large language models are fluent about proteins in the abstract but were not trained to read sequences. BetaDescribe closes that gap by continuing a general-purpose language model's training on protein sequence paired with curated English description, so it learns to map between the two domains directly.

The system is a pipeline, not a single network. A large generator proposes candidate descriptions; three small classifiers, the validators, independently predict simple properties of the same sequence; and a judge rejects candidates that contradict the validators. The generator and validators are released as downloadable checkpoints; the judge is an external commercial LLM accessed through an API key, the one component a user cannot run locally.

#Key Features

• Free-text output, not a label set: The model writes UniProt-style descriptions rather than assigning Gene Ontology terms or EC numbers, so it can express function for which no controlled-vocabulary term is a good fit.
• Generate-then-validate architecture: Small ESM-2-based classifiers predict taxonomy, subcellular localization, and enzymatic activity from the same sequence, and a judge discards generated descriptions that conflict with those predictions.
• Multiple ranked alternatives: Candidates that survive the judge are clustered by character n-gram similarity, and one representative from each of the three largest clusters is returned, ranked by cluster size — surfacing genuine ambiguity rather than a single confident guess.
• Homolog-free in-silico mutagenesis: Because the generator scores descriptions from sequence alone, mutating residues and re-describing the protein identifies regions important to function without requiring an alignment.
• Works where BLAST returns nothing: Evaluation centred on test proteins with no significant sequence similarity to the training set — the regime annotation transfer cannot reach.

#Technical Details

The generator is a decoder-only model initialized from LLaMA-2 — seven billion parameters, 32 layers, 32 attention heads, hidden size 4,096 — and trained in three consecutive stages on roughly 120 billion additional tokens. Stage one mixes 30% English text from RedPajama with 70% UniRef90 protein sequences so biological vocabulary enters without losing fluency. Stage two pairs proteins with their cognate UniProt descriptions in both directions, sequence-to-text and text-to-sequence. Stage three trains description prediction from sequence alone. The three validators are each fine-tuned from the 150-million-parameter ESM-2 base model, predicting higher-level taxonomy, multi-label subcellular localization across 388 categories, and a binary enzymatic-activity flag.

Evaluation partitioned a 2.5-million-protein test set by BlastP E-value against the training data. For the 189 Category 1 proteins with no BlastP hit at all, the top prediction matched the true UniProt description exactly for 27, with a mean ChrF of 0.43 and cosine similarity of 0.66; excluding proteins whose exact sequence also appeared in training, ChrF settles at 0.34. For the 172 Category 2 proteins whose best hits are statistically insignificant, the first prediction reached cosine similarity 0.58 against 0.48 for the best BlastP hit. Where close homologs exist, annotation transfer from BlastP remains the more accurate route — the authors say so directly — but agreement between the two signals is itself informative, since BlastP predictions congruent with BetaDescribe are measurably more accurate than incongruent ones.

#Applications

The natural users are groups annotating proteins from poorly characterized organisms — viral and phage proteomes, environmental metagenomes, non-model eukaryotes — where many predicted ORFs return no useful BLAST hit and are left as "hypothetical protein". The paper works through several such cases, including envelope and surface proteins from fish and reptile viruses. The in-silico mutagenesis mode adds a second use: scanning a sequence for regions whose perturbation most changes the generated description, nominating candidate functional sites for experimental follow-up without an alignment or a structure.

#Impact

BetaDescribe is an early entry in a line of work treating protein annotation as a translation problem rather than a classification one, and it is candid about where that helps. The gains are concentrated in the homolog-free regime; where BLAST works, BLAST wins. Two constraints bear on adoption: the judge stage depends on a closed external API, so the fully released pipeline is the generator and validators alone, and inference was tested on single GPUs with roughly 46–48 GB of memory. All four checkpoints carry a CC-BY-NC-4.0 licence, the peer-reviewed PNAS article is CC-BY-NC-ND, and the code repository carries no licence at all.