Loading…
4-bit QLoRA fine-tunes of ESM-2 for per-residue protein binding site prediction, released as a checkpoint family spanning 8M to 650M parameters.
Predicting where a protein binds other molecules — metal ions, nucleic acids, small-molecule ligands, or other proteins — is central to drug discovery and enzyme engineering, and has historically required solved three-dimensional structures or painstaking co-crystallography.
QBind, from independent researcher Amelie Schreiber, predicts binding residues from amino acid sequence alone by fine-tuning ESM-2 with QLoRA: low-rank adapters trained against a backbone quantized to 4-bit precision. It was introduced in a November 2023 bioRxiv preprint alongside ESMBind, the full-precision LoRA variant of the same idea, and the two are distributed as separate HuggingFace collections.
The distinction that matters in practice is reach. Quantizing the frozen backbone is what makes the 650M-parameter ESM-2 tractable to fine-tune on a single consumer GPU, and the released checkpoints reflect that: this family extends to 650M, while the full-precision collection tops out at 35M. If the goal is the strongest available representation rather than the cheapest, this is the variant that has one.
ESM-2 serves as a frozen sequence encoder with LoRA adapter matrices injected into the query, key, and value projections of the attention mechanism, and a lightweight token-classification head projecting per-residue hidden states to class logits. Only the adapter parameters and the head are updated. For the quantized variant the backbone is additionally cast to 4-bit NF4 using bitsandbytes, with double quantization compressing the quantization constants themselves.
Binding site prediction is framed as per-residue binary token classification. Datasets are compiled from UniProtKB/Swiss-Prot annotations and the Protein Data Bank. Because binding residues are a small minority of any protein, training uses class-weighted cross-entropy or oversampling to address the imbalance. Training uses AdamW with a cosine schedule and modest weight decay.
Benchmarks in the preprint report F1, Matthews correlation coefficient, and AUROC. The quantized variants land within a few points of full-precision LoRA across the tasks evaluated while cutting memory consumption by roughly 60–70%, which is the central result: at these model sizes, quantization costs almost nothing in accuracy.
The practical use is early-stage triage. In drug discovery, binding pockets can be identified on targets emerging from genomic or transcriptomic studies before any structure is solved. For structural biologists, sequence-based predictions prioritize which residues to mutate in alanine-scanning campaigns mapping interface contacts. Researchers studying host-pathogen interactions can predict interaction surfaces on newly sequenced pathogen proteins to generate mechanistic hypotheses. Because inference is fast and cheap, the checkpoints also work as low-resource components inside larger annotation pipelines, supplementing database lookups for proteins with little prior characterization.
This was an early demonstration that the parameter-efficient fine-tuning methods that reshaped natural language processing transfer directly to protein annotation, with the quantized variant establishing that 4-bit training carries no meaningful accuracy penalty at these scales. Open MIT-licensed checkpoints on HuggingFace lowered the barrier for non-specialist labs to deploy binding site prediction without maintaining custom codebases.
Two limitations are worth naming. The models have not been benchmarked head-to-head against the strongest structure-based predictors on shared held-out sets, so the residual gap against methods using three-dimensional coordinates is unquantified. And the preprint had not received formal peer-review publication as of early 2026, though the methodology has been cited and built upon in later work on protein annotation with language models. Post-translational modification prediction, sometimes attributed to this collection, is in fact a separate release: the ESM-PTM collection carries its own LoRA and QLoRA checkpoints for that task.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.