Cell2Text

Single-cell foundation models such as Geneformer and scGPT learn transferable representations of gene expression, but their downstream use almost always ends in a classification head. That head fixes the label space in advance: a new tissue, disease context, or rare cell state requires assembling annotations and fine-tuning a new classifier, and the output is a single categorical token where a biologist would want a sentence explaining what the cell is and what it is doing.

Cell2Text addresses this by treating cell annotation as a generation problem. A frozen single-cell encoder produces contextual gene-level embeddings, a trained adapter projects them into the token embedding space of an instruction-tuned language model, and the language model writes a structured description covering cell type, tissue of origin, disease association, donor development stage, and active biological pathways. Because the output is free text rather than a logit over a fixed vocabulary, the same checkpoint can characterize cells from donors and contexts it never saw during training.

The model was introduced in a September 2025 arXiv preprint by researchers at Ecole Polytechnique, the National Technical University of Athens, and MBZUAI. It follows the same encoder-adapter-decoder recipe the group applied to proteins with Prot2Text, and sits alongside Cell2Sentence, which instead serializes a cell as a ranked list of gene names, and ChatNT, which does conversational multimodal work on nucleotide sequences.

#Key Features

• Gene-level rather than pooled conditioning: Instead of compressing a cell into one embedding, the adapter forwards the full sequence of per-gene embeddings to the decoder, preserving transcriptional detail that mean-pooling discards.
• Open-vocabulary output: Descriptions are generated as text, so cell types, tissues, and diseases outside a predefined label set can be expressed without adding a new classification head.
• Ontology-aware evaluation: Predictions are scored with a Personalized PageRank similarity over the Cell Ontology graph, crediting near misses such as "T cell" for "CD4-positive T cell" instead of treating them as total failures.
• Purpose-built multimodal corpus: The authors assembled one million CELLxGENE Census cells paired with structured descriptions derived from Cell Ontology definitions and pathway enrichment, split by donor to prevent leakage.
• Two decoder scales and two tuning regimes: Variants use a 1B or 4B instruction-tuned decoder, trained either with full fine-tuning or with LoRA, letting deployments trade compute against generation quality.

#Technical Details

The encoder is Geneformer-V2-316M, kept frozen throughout training, with a 4,096-gene context window that covers the expressed genes of nearly any cell. The adapter is a two-layer feedforward network with a non-linear activation; its outputs are L2-normalized before entering the decoder. Decoders are Meta-Llama-3.2-1B-Instruct or Gemma3-4B-it, prompted with a fixed system message plus the projected gene embeddings. LoRA variants apply rank-256 adapters (alpha 512) to the decoder's self-attention only. Training ran two to three epochs on eight V100 or A100 GPUs with Adam.

Training data comprises 1,000,000 cells from 7,331 donors spanning 783 cell types, 347 tissues, and 128 disease conditions, with an 80/10/10 donor-level split. Reference descriptions combine ontology definitions with the two most enriched of 34 MSigDB Hallmark pathways scored by pySCENIC. Parsing labels back out of the generated text, Cell2Text-Gemma-4B reaches 77.83% cell type accuracy versus 67.26% for a linear head on frozen Geneformer embeddings and 50.7% for LightGBM; the Llama-1B variant leads on tissue (73.35%) and disease (77.84%). PageRank similarity is 85.62% versus 80.62% for the Geneformer head, and BioBERT BERTScore F1 exceeds 93.9. On multi-label pathway identification the generative models beat the linear head but trail a 34-classifier LightGBM ensemble on subset accuracy.

#Applications

The practical draw is annotating atlases without first committing to a label taxonomy. A lab profiling a new tissue or disease cohort can pass cells through a single checkpoint and get readable summaries naming identity, context, and active programs — useful for triaging clusters before manual curation, describing states that fall outside a reference atlas, and making model output legible to collaborators who do not work with embeddings. Generated text is also searchable and comparable, which suits large-scale screening.

#Impact

Cell2Text is an early demonstration that pairing a frozen single-cell encoder with a language decoder can beat the specialized classifier built on the same encoder, suggesting that the generation objective extracts more from the representation than a discriminative head does. Several caveats bound the result. The preprint has not been peer reviewed and the comparisons are the authors' own. Reference descriptions are templated from CELLxGENE metadata and automated pathway scoring rather than written by experts, so high lexical and semantic similarity partly reflects that structure. The one-million-cell training set is small relative to the corpora behind the encoders it builds on. Code is on GitHub but carries no license, and no trained weights have been released, so reproducing the model currently means retraining it.