LambdaFold

A single-sequence structure predictor can look accurate for two very different reasons: it has learned how proteins fold, or it has already seen something close to the target. Nearly 87% of the CASP14 and CAMEO benchmark proteins share at least 25% sequence identity with a protein in the ESMFold training set, and ESMFold's accuracy on them tracks how familiar each sequence is — its TM-score correlates with sequence perplexity at −0.63. Separating the two explanations requires training data held explicitly apart from the benchmark.

LambdaFold, from the Kihara Lab at Purdue University, is built to be that predictor. It folds a protein from a single sequence with no multiple sequence alignment, using the Prot-LAMBDA language model as its sequence and pair encoder and a compact folding trunk called DistFormer in place of a deep Evoformer stack. The premise: if inter-residue distance is modeled explicitly and hierarchically rather than left to emerge from depth, the network need not be deep. DistFormer refines the distogram from coarse to fine across three sequential two-layer blocks — 4, then 16, then 64 distance bins spanning 2.3–21.6 Å — and a shared invariant point attention (IPA) module reads out a full-atom structure after every block.

Trained on a deliberately small, non-redundant corpus, LambdaFold is far less sensitive to sequence familiarity: its own TM-score/perplexity correlation is −0.27. A variant, LambdaFold-RAG, adds retrieval — templates pulled from PDB70 are fused into the predicted distogram to steer a further round of refinement.

#Key Features

• Progressive distogram refinement: Three two-layer DistFormer blocks predict inter-residue distances at 4, 16 and 64 bins in turn, each frozen before the next is trained. Mean contact F1 rises across the three stages from 0.48 to 0.59 to 0.62.
• Regional and sparse attention: Borrowed from vision transformers, regional attention over non-overlapping 8×8 windows captures intra-domain geometry while sparse attention over a sampled residue grid captures inter-domain contacts.
• Template fusion by triangular cross-attention: In LambdaFold-RAG, two blocks apply triangular self-attention to the predicted distogram and then triangular cross-attention against a retrieved template distogram, gated to keep the model's own prediction wherever the template is silent.
• Distance feedback instead of a bias term: The fused distogram drives an IPA loop that recomputes a distogram from the current coordinates and feeds the residual distance error back as the pair representation, giving template geometry a direct role in refinement.

#Technical Details

DistFormer is six layers against ESMFold's 48, and 58M parameters against ESMFold's 686M folding trunk, with sequence and pair embedding widths growing across blocks from 512/192 to 640/224 to 768/256. Training used the Prot-LAMBDA corpus of about 409,000 structures (98,908 PDB chains released before January 2020, plus 310,267 AlphaFold DB models), each stage trained for 30 epochs with AdamW at batch size 1,024, for roughly 7,500 GPU-hours on four NVIDIA A100s. The structural loss follows AlphaFold 2 as implemented in OpenFold, with the distance term formed as a product of Huber and cross-entropy losses.

Across the combined 244 CASP14 and CAMEO targets, with no recycling, LambdaFold reaches a mean TM-score of 0.71 and LDDT of 0.74, against ESMFold's 0.78 and 0.81. On the 30-target subset held under 25% identity to the ESMFold training set the gap closes: mean TM-scores of 0.51 against 0.52, with LambdaFold ahead on LDDT at 0.66 against 0.65. On the 90 targets with more than 75% template coverage, LambdaFold-RAG lifts mean TM-score from 0.80 to 0.85, improving 56 of them and rescuing several outright failures — target 7eqs_A moves from 0.21 to 0.79.

#Applications

LambdaFold addresses the cases where alignment-based folding has least to work with: orphan proteins, de novo designs, and fast-evolving viral sequences, where a homology search returns too few relatives to build a usable alignment. Its small footprint suits groups without large GPU allocations; the repository ships notebooks that produce a contact map, a distogram, or a predicted structure for one sequence in under ten minutes. LambdaFold-RAG fits the case where a related structure exists in the PDB but sequence identity is too low for conventional homology modeling.

#Impact

LambdaFold is a preprint awaiting peer review, and the authors are candid that it does not beat ESMFold overall; the claim is parity on targets ESMFold has not effectively seen before, at a fraction of the folding-trunk size. The more consequential contribution may be diagnostic: holding training data to 25% identity and reporting accuracy against sequence perplexity quantifies how much of a single-sequence predictor's benchmark score rests on training-set familiarity. The limitations are stated plainly — the shallow trunk limits recycling, the small training set hurts on sequences far from the training distribution, and the evaluation is entirely computational. Code and weights are released under GPL-3.0, with the authors inviting anyone needing different terms, commercial use included, to contact them.