Loading…
Single-cell foundation model pretrained on 21 million mouse scRNA-seq profiles, with ortholog conversion extending its use to human transcriptomes.
No providers recorded yet. Browse all providers
Analyzing a mouse experiment with a human-trained single-cell model requires translating it first: every mouse gene is mapped onto its human ortholog, and roughly 15% of mouse genes have no such mapping and simply drop out of the input. Whatever mouse-specific regulation those genes carry is lost before the model sees the cell. Since the mouse is the workhorse mammalian model — with transgenic, knockout and knockin tooling that is ethically unavailable in humans — that translation tax falls on a large share of all single-cell work being done.
Mouse-Geneformer removes it by rebuilding the model in the mouse's own vocabulary. Keita Ito and colleagues at Chubu University, the National Institute for Basic Biology and the University of Tsukuba assembled mouse-Genecorpus-20M — 20,630,028 single-cell transcriptomes from healthy wild-type mice, spanning organs from embryonic through adult stages — and pretrained the Geneformer architecture on it from scratch. It is not a fine-tune of the human checkpoint but an independent model with a mouse gene vocabulary, released as a bioRxiv preprint in September 2024 and published in PLOS Genetics in March 2025.
The same ortholog trick then runs in reverse. Converting human gene names to their mouse homologs lets Mouse-Geneformer analyze human data, and after fine-tuning it matched the human Geneformer on cell-type classification — evidence that the rank-value-encoding architecture captures genetic architecture conserved across species, and a route to modeling non-model organisms for which no 20-million-cell corpus will ever exist.
The architecture follows the original Geneformer with three deliberate changes. It is a BERT-style transformer encoder with 6 encoder blocks, 4 attention heads, 256-dimensional embeddings and a 2048-token maximum input, trained with a masked-token objective over rank-value-encoded cell sentences in which 15% of tokens are masked. The modifications are SiLU in place of ReLU, ten pretraining epochs instead of three, and a cosine rather than linear learning-rate schedule; dropout is 0.02 and the optimizer is AdamW. Pretraining ran on 8 NVIDIA V100 GPUs with 32 GB each and took approximately two days. A larger 12-layer variant was also released.
Across twelve mouse organs, fine-tuned Mouse-Geneformer averaged 96.73% cell-type classification accuracy and never fell below 93%, against 72.95% for the autoencoder-based scVAE and 66.34% for the graph-neural-network scDeepSort. Pretraining accounted for up to 8.22 percentage points of that, measured on E9.5 whole-embryo data. On human data after ortholog conversion it reached 95.59% on thymus, 99.97% on cerebral cortex and 87.82% on breast — within 0.01 to 0.30 percentage points of the human Geneformer on the same splits.
The model gives mouse groups the transfer-learning workflow that human single-cell research already had: annotate cell types in a new organ or rare-cell dataset by fine-tuning on a few thousand labeled cells, then screen candidate disease genes computationally. In published disease models it recovered Slc12a3 as the deletion that most shifts normal kidney cells toward diabetic nephropathy, Slc35b1 as the strongest reversal target in UMOD nephropathy, and Apoe, Fth1, Itgax and Cst7 in COP1-knockout microglia — each corroborated by prior in vivo work. The cross-species path extends the same tooling to species without large single-cell corpora.
Mouse-Geneformer is the clearest demonstration that Geneformer's rank value encoding and transfer-learning recipe are not human-specific, and the paper frames the procedure as a template any species with enough scRNA-seq data can follow. The authors are candid about the limits. Cross-species agreement held for a myocardial infarction model but was only partial for COVID-19, which they attribute to laboratory mice not being susceptible to SARS-CoV-2 — a reminder that species-specific traits need species-specific models. In zero-shot mouse classification after ortholog conversion, scGPT outperformed Mouse-Geneformer on nine of twelve tasks, which the authors attribute to its GPT-style architecture. Strain composition of the corpus is also uncertain: most data comes from C57BL/6 or close relatives, but public strain annotation is sparse, so results in genetically diverse contexts warrant care.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.