Multimodal transformer that embeds mouse brain cells from RNA, chromatin accessibility, or both, tokenizing genes and peaks by their DNA sequence.
No providers recorded yet. Browse all providers
Single-cell foundation models usually represent a gene either as an arbitrary vocabulary index or as the protein-language-model embedding of the protein it encodes. Neither extends to an accessible stretch of chromatin, which has no protein product, so a cell measured by snATAC-seq cannot easily be compared with one measured by snRNA-seq. Bloom addresses this by giving every genomic feature, gene or peak, the same kind of token: an embedding of its underlying DNA sequence from a genomic language model, plus its relative position on the chromosome and its measured state in the cell. Because the token does not depend on the assay, one encoder can embed a cell from RNA, from ATAC, or from both.
Bloom was developed at the Allen Institute by Rohan Gala, Ian Convy, Yeganeh Marghi and colleagues in the labs of Uygar Sümbül and Hongkui Zeng, with collaborators at Amazon Web Services. The preprint appeared in November 2026 with inference code and the evaluated checkpoint, hyacinth. It is unrelated to BigScience's BLOOM language model.
Each training step draws two random views of the same cell, subsets of its detected genes and accessible peaks that may come from different modalities, and trains the model to embed both alike. Modality becomes a nuisance variable, and the random sampling acts as data augmentation: Bloom was pretrained on roughly 100-fold fewer cells than comparable pretrained cell models.
Each token combines a 768-dimensional PCA projection of the feature's Evo 2 embedding, a relative genomic-position encoding, and its binarized measurement. A first attention layer with 8 sequence, 8 position and 4 standard heads contextualizes the tokens. A Perceiver block then compresses them into 256 learned latents of 1,024 dimensions, which pass through 8 self-attention layers and are mean-pooled into a 1,024-dimensional cell embedding. Two heads train the model: a Bernoulli decoder that predicts whether each of about 8,000 query genes is detected, and a VISReg alignment head. Context length is up to 8,192 features per view. The released checkpoint has 124 million parameters (the paper rounds to about 130M) and was trained for 10 days on 16 NVIDIA H200 GPUs on 292,221 unpublished Allen Institute adult mouse brain multiome cells (paired snRNA-seq and snATAC-seq), capped at 1,000 cells per subclass across 307 subclasses. Scaling to 1.25 million multiome cells, or adding about 570,000 unpaired RNA cells, did not improve accuracy.
All evaluations use frozen embeddings. Linear classifiers reach 91% adjusted balanced accuracy over 315 subclasses on held-out multiome cells, and 94% on unseen whole-brain scRNA-seq and snRNA-seq datasets, ahead of UCE, State's SE-600M and TranscriptFormer. Bloom leads on visual-cortex Patch-seq and ties SE-600M on aging mouse brain. Having seen no human data, it trails the human-trained baselines by under 5% on human cortical snRNA-seq and does worse on fine-grained SEA-AD Alzheimer's supertypes.
Bloom serves brain-atlas and neuroscience groups who need one embedding space across assays: annotating new mouse brain datasets against the Allen whole-brain taxonomy, integrating scRNA-seq, snRNA-seq, Patch-seq and snATAC-seq cells, and labeling ATAC-only experiments from RNA references. Its gene–peak attention scores rank candidate regulatory elements for a target population; 15 of 30 peaks around Pvalb were attended significantly more in Pvalb interneurons than in L5 extratelencephalic neurons, pointing to candidates for cell-type-selective genetic access. The released package embeds AnnData objects on CPU or GPU, and accepts human RNA only through a one-to-one ortholog mapping onto the mouse reference.
Bloom shows that tokenizing by DNA sequence lets chromatin accessibility join a pretrained cell model on equal terms with expression. A small, balanced multiome corpus matched or beat much larger RNA-only models on brain cell typing. That more cells of the same kind did not help supports the view that diversity matters more than raw cell count for single-cell training data. The limits are clear: training covers only adult mouse brain, reference tables exist only for mm10, the training data is unpublished, and the weights are CC BY-NC 4.0 (non-commercial) while the inference code is MIT. The gene–peak associations are untested computational hypotheses, and the work is a preprint that has not been peer-reviewed.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.