Loading…
Protein language model for variant effect prediction and de novo sequence design, conditioned on Gene Ontology embeddings of molecular function.
No providers recorded yet. Browse all providers
Predicting how a mutation changes a protein's fitness has long been a family-specific exercise: fit a model to the target's own homologs, then score variants under it. That works when the family is large and diverse, and it degrades exactly where help is most wanted. Alignment-based predictors such as EVmutation and DeepSequence estimate residue couplings from a multiple sequence alignment and run short of signal when homologs are scarce; alignment-free models trained on the same family meet that shortage as a shortage of training data. Func2Seq changes the conditioning variable from family membership to molecular function, so a thinly populated family can inherit statistics learned from unrelated families annotated with related functions.
The conditioning signal is a learned Gene Ontology embedding rather than a flat tag. Each Molecular Function term's definition and description are encoded with BioBERT, and those text features are propagated over the ontology's directed acyclic graph by a two-stage sequential graph convolution that passes messages bottom-to-top and then top-to-bottom instead of isotropically, so the hierarchy survives into the embedding space. Func2Seq is then a transformer decoder for P(seq | func) whose token representations cross-attend to the set of function embeddings its target family carries. A single global meta-pretraining run produces the released checkpoint, which is fine-tuned per family for variant scoring and used unchanged for function-conditioned generation.
Func2Seq comes from Yang Shen's group at Texas A&M University, published in Biophysical Journal in 2026 alongside Func2Prot, its joint sequence-structure extension.
The decoder stacks four layers by default (six in the Func2Seq-large configuration), each combining self-attention over generated tokens, cross-attention onto the function embedding set, and a feed-forward block, with 12 attention heads and 768-dimensional hidden states. The function encoder holds the same 768 dimensions and is trained separately with an InfoNCE-style contrastive objective whose positive pairs are upweighted by ancestor-set overlap in the ontology, over Molecular Function terms from the go-basic release of 2020-12-08 mapped to families through Pfam2GO and GODM. Meta-pretraining runs on Pfam-34 RP15 filtered to 8,772,255 sequences — discarding non-natural amino acids, families without GO annotation, and sequences over 800 residues — with a scaling variant on 97,412,225 Pfam-35 RP75 sequences paired with the six-layer model.
On the 35-task variant effect benchmark of Shin et al., covering 33 proteins, Func2Seq beats the alignment-free baseline in 29 of 35 tasks and DeepSequence in 18 of 35, raising mean Spearman correlation by 0.041 and 0.053 respectively (one-sided t-test p = 6.2 × 10⁻⁶ and p = 2.9 × 10⁻³), and scores higher than ESM3 on all 35. The ablation splits the credit: warm starting from the meta-pretrained checkpoint beats cold-start fine-tuning of the same architecture in 32 of 35 tasks, a mean gain of 0.026, while fine-tuning remains indispensable — pretraining alone surpasses a family-specific profile HMM in only 9 of 35 tasks.
The factor analysis says where to reach for it: gains over the alignment-free baseline track training-family diversity (Spearman 0.303) and gains over DeepSequence track effective family size (0.286), concentrating the benefit on families that are small, low-diversity, or short of homologs. On enzyme fitness classification for chorismate mutase and malate dehydrogenase, Func2Seq outperforms ProteinGAN and the small, large and extra-large ProGen2 variants. For design, fine-tuning on 18,740 filtered ARID DNA-binding domain sequences (Pfam PF01388) produced candidates in the twilight zone of 20–30% global maximum sequence identity to natural ARIDs that still held above 40% local identity across the L1 and L2H5 binding regions; a representative such design folded by AlphaFold 2 reached a TM-score of 0.645 against the reference fold.
What generalizes beyond this benchmark is the conditioning signal: encoding a function ontology as text plus topology gives a generative protein model a specification with real structure in it, and the ablations make the resulting transfer measurable rather than asserted. The limits are equally explicit. Every reported score depends on family-specific fine-tuning, so the released pretrained checkpoint is a starting point rather than a zero-shot predictor; conditioning reaches only families present in the Pfam-to-GO mapping; the design evidence rests on one family; and no designed sequence was characterized experimentally.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.