A 3.15B-parameter single-cell foundation model that represents each gene by its mRNA sequence embedding, pretrained on 85M human and mouse cells.
No providers recorded yet. Browse all providers
Most single-cell foundation models treat a gene as an entry in a lookup table: ACTB is one fixed token index, and whatever the model learns about it is tied to that index. The table works within one species and one annotation, and breaks at the edges. A lemur gene without a curated human ortholog has no row, a bacterial transcript has no identifier at all, and a patient's variant allele is indistinguishable from the reference because both share one ID. LucaCell drops the lookup table. It describes each gene by what the gene is, an embedding of its mRNA sequence, so any transcript that can be sequenced can be placed in the model's input space.
The gene embeddings come from LucaOne, the same group's unified nucleic-acid and protein language model, which is held frozen throughout. Each gene's sequence is passed through LucaOne and mean-pooled into a 2,560-dimensional vector. That vector is paired with the cell's binned expression level for the gene, and a transformer encoder learns how genes co-vary within cells. LucaCell was developed by the Token Foundry group at Alibaba Group with BGI Research, the University of Chinese Academy of Sciences, Zhejiang University, Westlake University and Xiamen University. It was posted to bioRxiv in September 2026 and has not yet been peer reviewed.
LucaCell is a 40-layer transformer encoder with 40 attention heads, a hidden size of 2,560, a feed-forward size of 10,240 and rotary positional embeddings. The released checkpoint holds about 3.15 billion parameters. Expression is discretized into 50 non-zero bins plus a special token for unexpressed genes. Each cell is represented as up to 1,000 expressed genes plus sampled zero-expression genes, 1,200 tokens in total, ordered by genomic coordinate. Pretraining used a masked gene modeling objective, predicting masked expression bins from cellular context. The corpus held about 85 million scRNA-seq profiles (67 million human and 18 million mouse) from 62 tissues and 18 pathological states. Training ran on 8 NVIDIA A100 GPUs in bfloat16, and the released checkpoint is from step 6.4 million.
The authors report the following results. In the kidney cell-type task, accuracy was 0.937 on human scRNA-seq and 0.961 on lemur scRNA-seq, and a mouse cohort was held out entirely as a test set. Microbial taxonomic classification over 54 classes from a human gut dataset reached 84.1% accuracy. LucaCell's read-based clusters for Pseudomonas aeruginosa agreed with alignment-based clustering (ARI 0.864). On Adamson CRISPR perturbations it had the lowest MSE among Geneformer, scGPT and scFoundation, though scGPT had a higher delta Pearson correlation (0.913 vs 0.851). The authors note that these baselines differ in architecture, scale and corpus, so the gains cannot be attributed to sequence-based gene representation alone.
LucaCell suits single-cell work where fixed gene identifiers break down. Examples include annotating cells from non-model organisms with poor ortholog coverage, integrating RNA and chromatin accessibility data, and profiling bacterial single-cell transcriptomes without a reference genome. It also covers studies of how genotype or infection shapes cell state. Researchers can extract frozen cell embeddings for clustering and visualization, or use the checkpoint to initialize task-specific heads, as the paper does for classification and regression.
LucaCell extends the Luca family from sequence modeling into cellular modeling, showing that a nucleic-acid language model can act as the gene vocabulary for a cell model. This joins models such as UCE and TranscriptFormer in the move away from species-locked gene tokens. LucaCell differs by grounding genes in transcript sequence, which also lets it reach prokaryotic and viral transcripts. Pretraining code, inference code, the checkpoint and downstream data are all public. The pretraining corpus is served from a temporary IP-address link, and the authors state that they plan to deposit it in a public repository.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.