Antibody language model pretrained on 31.8 million fully human heavy-chain-only antibody sequences from transgenic mouse repertoires.
No providers recorded yet. Browse all providers
A conventional antibody buries roughly 800 square angstroms of its heavy chain against a light chain partner. Take the light chain away, as heavy-chain-only antibodies (HCAbs) do, and that former interface becomes solvent-exposed framework — a surface under selection pressures that no paired repertoire ever experiences. Antibody language models trained on conventional repertoires have seen almost none of these sequences, so the contextual preferences they learned for framework and loop positions were fitted on a different structural problem.
HCAbLM is Nona Biosciences' response: an encoder pretrained from scratch on a fully human HCAb repertoire rather than on paired or general antibody data. Before training anything, the authors compared 1,429 mature functional HCAbs against 3,410 publicly assay-positive conventional human VH domains under source-held-out weighting, and found a reproducible distributional shift concentrated in CDR1 and CDR2, CDR3 architecture, and a restricted framework region rather than broad framework remodeling. That model-independent difference is the argument for repertoire-specific pretraining.
The pretraining corpus is 31,822,392 globally unique HCAb VH amino-acid sequences from 73 independently immunized Harbour Mice raised against eight distinct antigens, sequenced by single-cell full-length immunoreceptor libraries. The scope matters: this is a model of the fully human HCAb format as produced by a transgenic mouse platform, not a general antibody model and not a paired VH/VL model. It is also distinct from camelid VHH modelling such as VHHBERT, since camelid and fully human HCAb domains come from different germline backgrounds and selection histories. The work is a September 2026 bioRxiv preprint, not peer reviewed.
HCAbLM-300M is an ESMC-style encoder-only transformer with 366.19 million parameters, trained de novo on NVIDIA H20 GPUs without initialization from pretrained ESM Cambrian weights, using the native ESMC tokenizer. The selected checkpoint consumed 15.2 billion amino-acid tokens at an estimated 3.34 × 10^19 training FLOPs. Model-size and checkpoint selection used only a frozen 80,000-target diagnostic bank scored by a region-weighted predictive statistic that combines pseudo-cross-entropy at NGL and true-CDR3 positions; the observed plateau fell at 0.400–0.407.
On per-residue prediction loss, the model diverges from conventional antibody models most clearly in CDR3: HCAbLM assigns lower standardized loss to HCAb than to human VH CDR3 residues (−0.651 s.d.), while IgLM assigns higher loss (+0.186 s.d.) and AbLang-2 runs in the same direction as IgLM. The developability results come from Nona's in-house assay data, evaluated as grouped nested cross-validation plus a retrospective cross-project comparison rather than public benchmarks. HCAbLM reached AUROC 0.691 on the internal SEC purity classification endpoint, and the highest project-macro Spearman correlations for SEC purity (0.355) and HIC favorability (0.453); HCAb-adapted IgBERT led on internal HIC correlation and ESMC-6B on thermal transition midpoint. Because held-out project performance fed endpoint-level model comparison, those cross-project estimates are selection-affected.
The intended use is triage inside an HCAb discovery campaign: embedding candidate VH sequences and ranking them on chromatographic and thermal behavior before committing material to purification and assay time. Because the encoder stays frozen, a team with a few hundred measured molecules can fit a shallow head rather than a deep network. The same region-resolved compatibility scores support review of CDR and framework positions in single-domain formats reused as bispecific arms, CAR binders and antibody-drug conjugate carriers.
HCAbLM's contribution is the evidence that repertoire composition is itself a design variable for antibody foundation models: a 366M-parameter encoder trained on one specialized format can match or exceed billions-of-parameters general protein encoders on some properties of that format while losing on others, so format-specific pretraining buys region-selective priors rather than uniform superiority. The practical caveats are real and worth stating plainly. The pretraining repertoire is proprietary and cannot be redistributed, training code and weights are not planned for unrestricted public release, the endpoint advantages are endpoint-dependent and awaiting a prospectively locked evaluation set, and the whole result rests on a preprint that has not been certified by peer review.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.