CellWhisperer

"Show me tissue-resident T cells in the intestine" is a question a biologist can ask in one breath and a single-cell pipeline cannot answer at all. To get there conventionally you either assemble a marker gene list and score every cell against it, or you find a labelled reference atlas that already contains the label you are looking for and transfer its annotations. Both routes require you to know the answer's vocabulary in advance, and neither handles a query that mixes cell type, tissue, disease state and function in one sentence.

CellWhisperer removes that translation step by learning the correspondence between transcriptomes and the words biologists use to describe them. Its embedding model is a CLIP-style contrastive network: one encoder reads an expression profile, another reads a free-text annotation, and training pulls matched pairs together in a shared vector space. Once that space exists, a typed query is simply another point in it, and scoring every cell in a dataset against the query becomes a dot product. A second model — an open-weights LLM fine-tuned to accept those embeddings as input — turns the same representation into conversation, answering questions about a selection of cells rather than only ranking them.

The model comes from Christoph Bock's group at CeMM and the Medical University of Vienna, with collaborators at the St. Anna Children's Cancer Research Institute, and was released as a preprint in October 2024 and published in Nature Biotechnology in November 2025.

#Key Features

• Free-text search over expression data: Any English phrase becomes a per-cell "CellWhisperer score", so datasets can be queried by cell type, state, tissue, disease or biological function without a controlled vocabulary.
• Annotation without a reference dataset: Cell characteristics are predicted zero-shot from the shared embedding — no task-specific fine-tuning and no labelled atlas to transfer from.
• Transcriptome-aware chat: A fine-tuned Mistral 7B accepts CellWhisperer embeddings as multimodal input and discusses the selected cells, their marker genes and the biology behind them.
• Integrated into CELLxGENE Explorer: The chat box ships inside a modified CELLxGENE browser, so text queries, lasso selection and UMAP inspection happen in one interface.
• Weights and training data published: Checkpoints for the embedding and chat models, plus the full AI-curated training corpus, are downloadable from the lab's file server.

#Technical Details

Training data comprises 1,082,413 human RNA-seq profiles paired with textual annotations: 705,430 bulk profiles from GEO via the ARCHS4 uniform reprocessing, and 376,983 pseudo-bulk profiles derived from CELLxGENE Census single-cell datasets. In both cases an LLM condensed the raw sample metadata into a concise biological description. The embedding model encodes transcriptomes with Geneformer and text with BioBERT v1.1, projecting both through feed-forward layers into a 2,048-dimensional joint space trained with the CLIP objective. Text-to-transcriptome retrieval reaches a mean AUROC of 0.927. Zero-shot predictions on held-out data give AUROC 0.94 across 20 common Tabula Sapiens cell types and 0.91 across all 177, above 0.99 on ImmGen bulk immune profiles and the Asian Immune Diversity Atlas, 0.89 on a batch-effect-heavy pancreas meta-analysis, 0.82 across 229 disease subtypes and 0.75–0.87 for tissue of origin. The chat model adapts Mistral-7B-Instruct-v0.2 with a LLaVA-style adapter that maps embeddings into the LLM's token space, fine-tuned on 106,610 generated conversations. Variants substituting scGPT or UCE for Geneformer are also released.

#Applications

The primary use is exploratory analysis of a dataset the analyst does not yet know well: labelling clusters, locating cells matching a described phenotype, and interrogating a selection in conversation. The authors demonstrate discovery as well, using organ-name queries across a 95,092-cell meta-analysis of human embryonic development to recover known organ marker genes and at least ten additional candidates per organ. Users can process their own human raw-count matrices locally — roughly two hours per 10,000 cells on CPU, faster on GPU — or explore the hosted example datasets.

#Impact

CellWhisperer is a proof of concept that natural language can serve as the primary interface to transcriptome data, and its zero-shot cell typing matched single-cell foundation models explicitly fine-tuned for that task while beating marker-based annotation. Its limits are equally clear: training and inference cover human data only, disease-subtype prediction is far weaker than cell typing, CLIP-style scores are sensitive to how a query is worded, and the chat model's answers blend the embedding with the base LLM's prior knowledge, so claims it makes about specific genes warrant checking.