BCR-Mistral

BCR-Mistral is an unsupervised foundation model for B-cell receptor (BCR) sequences, built by downscaling the Mistral-7B decoder-only transformer architecture and pretraining it from scratch on antibody nucleotide sequences. It was developed by Graham Peet, Gregory Owens, Jeffrey Bennett, Arjun Krishnan, and Wendy Macklin at the University of Colorado Anschutz Medical Campus, and described in a bioRxiv preprint posted in July 2026.

The design choice that separates BCR-Mistral from most antibody models is that it operates on DNA rather than protein. Antibody representation learning is dominated by protein language models trained on translated amino acid sequences, but a rearranged immunoglobulin heavy chain (IGH) carries information that translation discards. The authors' rationale is that a nucleotide model captures everything available at the protein level while also learning genetic patterns that are not evident once a sequence is translated — the signatures of V(D)J recombination and somatic hypermutation that define a repertoire's history. On sequence embedding tasks, BCR-Mistral outperformed both BCR-specific and general-purpose protein language models, and it also beat a comparably trained protein language model built by the same group as a controlled comparison.

Pretraining used public, non-MS antibody data, deliberately disjoint from the multiple sclerosis (MS) classification work the model was built to support. That separation is what makes it a reusable checkpoint rather than a single-task predictor: downstream classifiers consume frozen embeddings from the pretrained model rather than fine-tuning it end to end. The preprint is not yet peer reviewed, and the authors state that code and model weights will be released on GitHub and Zenodo upon publication; neither is public yet.

#Key Features

• Nucleotide-level antibody modeling: Trains directly on IGH DNA sequences rather than translated protein, retaining recombination and somatic hypermutation signal that amino acid models cannot see.
• Three pretrained scales: Large (~500 million trainable parameters), medium (~300 million), and small (~100 million) decoder-only variants were pretrained under matched conditions; the large model substantially outperformed the smaller two and was used for every downstream task.
• Frozen-embedding transfer: Sequence and repertoire representations are extracted from the fixed pretrained model and fed to lightweight supervised classifiers, so new tasks require no retraining of the foundation model.
• Junction-specific companion model: A separate model trained only on junction sequences supplies embeddings for junction-level classifiers, isolating the hypervariable region that dominates antigen recognition.
• Benchmarked against protein language models: Evaluated against ESM-2 (650M) and Ab-RoBERTa on BCR embedding tasks.

#Technical Details

The three models are decoder-only transformers based on downscaled variants of the Mistral-7B architecture, pretrained with a next-token prediction objective. Training data came from the Observed Antibody Space (OAS) database maintained by OPIG, filtered to IGH sequences and transformed into 7-mers for tokenization, with sequences sampled according to Euclidean-distance criteria to control redundancy in the corpus. The learning rate was annealed from 1e-5 to 1e-7 over the course of pretraining, and each of the three sizes was trained for equivalent clock time so that scale comparisons reflected a fixed compute budget rather than a fixed number of steps.

The downstream evaluation corpus is a reanalysis of all publicly available RNA sequencing data from MS patients, yielding over 11 million BCR IGH sequences drawn from a range of tissue sources. Against this, embeddings from the 500M-parameter model supported classifiers that identify MS-associated BCR sequences, low-dimensional whole-repertoire representations that separate MS patient repertoires from healthy, infectious disease, and other autoimmune repertoires, and rankings that place previously characterized myelin-binding IgG sequences above controls.

#Applications

The model is aimed at researchers analyzing bulk antibody repertoires, where the practical problem is turning millions of raw IGH reads into representations that support downstream inference. Its demonstrated use is autoimmune disease characterization: flagging candidate disease-associated BCRs from blood, classifying patient disease state from repertoire structure, and prioritizing sequences for antibody discovery and functional follow-up. Because pretraining is disease-agnostic, the same frozen embeddings apply to any repertoire-level classification problem — infection, vaccination response, or other autoimmune conditions — for groups that already have BCR sequencing data and a phenotype label.

#Impact

BCR-Mistral is a concrete test of whether nucleotide-level modeling beats amino acid modeling for immune receptors, and its controlled comparison against a matched protein language model gives that question an answer rather than an assertion. The immediate caveats are practical: the work is a preprint awaiting peer review, the preprint carries a CC BY-NC-ND license, the authors have filed a patent application related to it, and the code and weights are pending publication, so the results cannot yet be reproduced or the checkpoints reused. Validation is computational throughout — the myelin-binding ranking uses antibodies characterized in prior work rather than new binding experiments — and the MS cohorts are assembled from archival public sequencing data rather than a prospective study.