Loading…
Generative protein sequence model built on Bayesian Flow Networks, sampling natural-like de novo sequences and inpainting any subset of residues.
No providers recorded yet. Browse all providers
An autoregressive protein language model writes a sequence from one end to the other, each residue conditioned only on what precedes it, while a masked model such as ESM-2 can fill a gap but has no principled way to produce a whole sequence from nothing. Neither ordering reflects how a protein works: a catalytic residue near the C-terminus constrains a loop two hundred positions upstream as tightly as it constrains its own neighbours. Discrete diffusion was meant to lift that constraint, but amino acids have no continuous scale to interpolate along, so its forward process must be defined over a finite set of categorical corruptions.
ProtBFN, from InstaDeep, takes a third route. A Bayesian Flow Network does not corrupt and denoise the sequence itself; it maintains the parameters of a categorical distribution at every position and refines them by Bayesian updates as progressively less noisy observations arrive. Those parameters are continuous even though the tokens they describe are not, which recovers the smooth denoising trajectory that makes diffusion effective without ever discretising the noise. Because the framework imposes no factorisation on the joint distribution, one trained network generates unconditionally or conditioned on an arbitrary subset of positions.
Released as a preprint in September 2024 and published in Nature Communications in April 2025, ProtBFN is the general-protein base model of the pair described in that work; the paper also fine-tunes it on antibody heavy chains to yield AbBFN, whose successor line is cataloged here as AbBFN2. Alex Graves, who introduced Bayesian Flow Networks, is an author on the paper.
ProtBFN is a 650M-parameter encoder-only transformer following the ESM-2 650M layout — 33 layers, 20 attention heads over a 1280-dimensional embedding, a 5120-dimensional MLP, rotary positional embeddings — with the initial token embedding replaced by a linear projection, since the input is a distribution over tokens rather than a token. Training data (UniProtCC) comes from the January 2024 UniProtKB release, keeping only entries at protein existence levels 1–3 and sequences shorter than 512 residues, for 71 million sequences. Pretraining ran for 250,000 steps at batch size 8192 on uniformly sampled data, then a further 250,000 steps on cluster-reweighted data, using 128 TPU v4 chips for roughly two weeks.
Sampling uses 10,000 steps, with perplexity and repetitivity filters discarding about 44% of raw draws. Across 10,000 generated sequences, 69.7% align to a UniRef50 cluster at ≥50% identity, and 65.7% produce an ESMFold structure that hits the CATH S40 database on both TM-score directions, against 25.3% for ProtGPT2 and 12.0% for EvoDiff; 80.4% of those hits share under 50% sequence similarity with their structural match. Fine-tuning on 195 million Observed Antibody Space heavy chains gives AbBFN, which recovers individual framework and CDR regions on par with AbLang2 and AntiBERTy despite never being trained to infill, and beats them when all four framework regions are masked at once.
The base model suits de novo design campaigns that need starting sequences spanning broad regions of fold space rather than variants of a known scaffold — the CATH analysis shows candidates covering alpha, beta, alpha-beta and irregular classes, including porins, transporters and enzymes, at single and multiple domain counts. The inpainting mode fits region redesign, where a scaffold is held fixed and a subset of positions resampled. Inference code and weights for both ProtBFN and AbBFN are released, runnable on CPU, GPU or TPU through a provided Docker image.
ProtBFN was an early demonstration that Bayesian Flow Networks transfer to biological sequence data, and InstaDeep extended the approach to the joint sequence-plus-metadata antibody model AbBFN2. Several limitations bound its use. All validation is computational, resting on predicted structures and database alignments rather than expressed and characterised proteins. The 512-residue training cutoff excludes longer proteins, and the antibody fine-tune covers heavy chains only. AbBFN trails the antibody-specific baselines on CDR-H3, which the authors attribute partly to the non-templated nucleotide addition that makes that loop intrinsically less predictable. The inference code is released under CC BY-NC-SA 4.0, barring commercial use, while the released weights carry CC BY 4.0.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.