Single-nucleus RNA-seq foundation model built on bidirectional Mamba blocks, reading all 19,306 genes per cell without dimensionality reduction.
No providers recorded yet. Browse all providers
scMamba is a pretrained model for single-nucleus RNA sequencing, developed at KAIST with a specific target: the snRNA-seq datasets generated in studies of neurodegenerative disease. Because brain tissue is too interconnected for whole-cell dissociation, brain transcriptomics relies on isolating nuclei from postmortem specimens — a source of unusually difficult data. Postmortem intervals vary and degrade sample quality, disease heterogeneity inflates variability across cohorts, and the small amount of mRNA in a single nucleus produces heavy dropout, leaving matrices dominated by zeros whose biological versus technical origin is ambiguous.
The architectural obstacle is length. A cell is a vector over tens of thousands of genes, and transformer-based single-cell models such as scBERT pay quadratic attention cost in that length, which is why most of them first restrict the input to a few thousand highly variable genes. That selection is itself unstable: which genes qualify depends on parameter choices and shifts between datasets, so different studies are not directly comparable.
scMamba replaces self-attention with bidirectional Mamba blocks. Selective state space models scale linearly in sequence length, which lets the model take the full gene vector as input, and the bidirectional variant removes the causal ordering that would otherwise be meaningless for genes.
Pretraining uses masked expression modeling: 15% of input embeddings are replaced with a mask token and the model regresses the held-out expression values under an L2 objective, with only nonzero entries masked since true and technical zeros cannot be distinguished. The corpus is roughly 1.6 million cells drawn from ten public snRNA-seq datasets; five further datasets (Lau, Leng, Smajic, Zhu, and an in-house 500,000-cell Jung dataset) are held out for downstream evaluation, spanning Alzheimer's and Parkinson's disease across prefrontal cortex, entorhinal cortex, superior frontal gyrus, substantia nigra, and hippocampus. All datasets were harmonized on Ensembl stable gene IDs, giving a shared vocabulary of 19,306 genes after filtering; the full annotated collection totals 2,408,023 nuclei from 461 patients. Classification is evaluated at three hierarchical levels — 8 major cell types, 72 subtypes, and 127 subclusters — where scMamba holds high F1 scores across all levels while baselines including scBERT, scHyena, and Seurat degrade on the finer ones. It also leads on in vivo doublet detection and on masked-value imputation, and imputation with scMamba visibly mitigates patient-level batch separation in UMAP embeddings.
The intended user is a neurogenomics group analyzing postmortem brain snRNA-seq: the model imputes dropped-out counts, flags doublets, assigns fine-grained cell identities, and supports differential expression testing between disease and control nuclei within cell types. Fine-grained annotation is where it earns its keep, since neuronal and glial subtypes in the brain often lack clean marker genes and are exactly the populations that turn out to be selectively vulnerable in neurodegeneration. Because scMamba is pretrained without labels, a lab can fine-tune it on a modest annotated subset of its own cohort rather than assembling a large reference.
scMamba is a demonstration that state space models are a practical fit for single-cell transcriptomics, where linear scaling in sequence length translates directly into the ability to keep every gene. Pairing that with continuous expression encoding removes two lossy preprocessing steps — gene selection and expression binning — that most transformer single-cell models still require. The scope is deliberately narrow: pretraining and evaluation are confined to human brain snRNA-seq in neurodegenerative disease, so performance on other tissues or on whole-cell scRNA-seq is not established. The code is released under Apache 2.0, but pretrained checkpoints are not posted publicly and must be requested from the authors, which limits reuse.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.