SC-MAMBA2

A human cell's transcriptome is a vector over roughly sixty thousand genes. Self-attention costs time quadratic in that length, so transformer-based single-cell models begin by discarding most of it — selecting a few thousand highly variable genes before the model sees anything. That selection is a modeling decision disguised as preprocessing: it depends on the dataset and the parameters chosen, and it settles in advance which gene-gene relationships the model is permitted to learn.

SC-MAMBA2, developed at the XtalPi Innovation Center in Beijing with collaborators at McGill University and Peking University, removes the constraint rather than working around it. It replaces attention with MAMBA2 state-space modules, whose cost is linear in sequence length, and keeps the entire 60,530-gene vocabulary of the CELLxGENE census as the input alphabet.

Doing so requires solving a mismatch first. Mamba is a recurrent, causal architecture that scans in one direction — the right inductive bias for text, the wrong one for a cell, where gene order carries no meaning. SC-MAMBA2's answer is the BiMamba block: the embedded gene sequence is reversed to form a flipped copy, both copies pass through weight-shared unidirectional Mamba modules, the flipped output is reversed back, and the two are summed, so every gene is conditioned on the full set rather than an arbitrary prefix. A Smart Padding rule flips only meaningful tokens, so the reversal introduces no artifacts.

#Key Features

• Whole-transcriptome input: The vocabulary spans all 60,530 genes indexed by the CELLxGENE census, so no highly-variable-gene filter stands between the count matrix and the model, and zero-expression genes are retained rather than dropped.
• Adapter-based task heads: Downstream tasks attach adapters to the pretrained backbone, so cell-type annotation, batch correction, multi-omic integration and perturbation prediction all reuse a single set of pretrained weights.
• Reverse perturbation: The fine-tuned model also runs the inference backwards, ranking which genes were perturbed to produce an observed state — the form the question takes in target discovery.
• Multi-omic transfer by modality token: For paired ATAC-seq or protein data the transcriptomic latent weights are retained, weights for the other modality are reinitialized, and a modality token plus adversarial domain adaptation maps both into a shared space.

#Technical Details

Pretraining uses masked expression modeling. Each gene contributes a name embedding plus an expression embedding over 51 discretized bins; the two are summed, 40% of expression values are masked, and a stack of 12 BiMamba layers regresses them under mean squared error. The corpus is the CELLxGENE census — over 57 million normal cells spanning brain (21.5M), blood (5.4M), lung (3.2M), heart (2.3M), liver (1.6M), kidney (1M), bone, pancreas, stomach and intestine. The v2 preprint reports over 625 million parameters, revised upward from the 150 million reported in v1. The whole-body checkpoint posted to Zenodo is a 288 MB file whose recorded configuration is 12 layers at 512 embedding dimensions with a 1,200-gene maximum input length — a smaller configuration than the headline figure, and the only checkpoint released.

The fine-tuned model reaches an AvgBIO of 0.836 on PBMC 10K batch integration, 7–12% above scVI, Seurat and Harmony and 1.83% above scGPT, and separates CD14+ from CD16+ naive cells on the 10x Multiome PBMC dataset where the compared methods do not. Cell-type annotation on hPancreas, Myeloid and Multiple Sclerosis matches or exceeds scGPT and TOSICA. On the Norman perturbation dataset — 105 single-gene and 131 two-gene perturbations — it holds at least 5% higher Pearson correlation with measured expression changes than scGPT, GEARS and scFoundation, the margin widest on two-gene perturbations where neither gene appeared in fine-tuning.

#Applications

The intended user runs standard single-cell workflows at scale: annotating cell types against a reference atlas, merging cohorts from different platforms without erasing biological variation, integrating RNA with ATAC or surface-protein measurements, and screening candidate perturbations in silico before committing to a CRISPR experiment. The reverse-perturbation head aims at target discovery, tracing a disease expression state back to the genes whose perturbation would produce it. Embeddings taken without fine-tuning cluster CD genes into groups aligned with distinct Reactome immune pathways, making the backbone usable for gene-network analysis too.

#Impact

SC-MAMBA2 makes a concrete case that state-space models earn their place in single-cell transcriptomics: linear scaling in sequence length converts directly into the ability to keep every gene, the preprocessing step most transformer-based single-cell models still cannot avoid. The reusable piece is the bidirectional adaptation — weight-shared forward and flipped scans with padding-aware reversal — which answers a problem any sequence model faces on unordered biological features. It should not be confused with two other catalogued state-space single-cell models by unrelated groups, GeneMamba and scMamba. SC-MAMBA2 remains a preprint and has not been peer reviewed, and its openness is mixed: the code is public but licensed CC BY-NC 4.0, barring commercial use, while the Zenodo checkpoint carries CC BY 4.0. Only the whole-body checkpoint has been released, though the repository's data pipeline also describes a pan-cancer variant.