TEDDY

TEDDY is a family of transformer-based foundation models for single-cell RNA sequencing (scRNA-seq) data, developed by researchers at Merck & Co. Where many single-cell foundation models optimize primarily for unsupervised representation learning, TEDDY is built to surface disease-relevant signal: its central goal is to classify cell states across donors and diseases the model has never seen, a setting that matters directly for drug discovery and target identification.

The model's distinguishing design choice is to fold large-scale biological annotations into pretraining as an explicit form of supervision, rather than relying on masked reconstruction alone. Alongside masked language modeling over gene expression, TEDDY is trained to predict ontology labels attached to each cell, grounding its representations in curated cell-type and disease structure. This annotation-aware pretraining is the mechanism the authors credit for stronger generalization on held-out donors and unseen diseases relative to self-supervised-only predecessors.

TEDDY was introduced in a March 2025 arXiv preprint (presented at the ICML 2025 Generative AI and Biology workshop) and updated in April 2026. The family spans three parameter scales — 70M, 160M, and 400M — and two tokenization strategies: TEDDY-G, which uses rank-based gene encoding, and TEDDY-X, which uses binned expression encoding. The TEDDY-G checkpoints are released on HuggingFace under an Apache 2.0 license.

#Key Features

• Annotation-aware pretraining: TEDDY combines masked language modeling with ontology classification, using curated biological annotations as supervision so representations are anchored to cell-type and disease structure, not expression reconstruction alone.
• Cross-donor and cross-disease generalization: The models are explicitly evaluated on classifying diseased versus healthy cells and identifying disease states in held-out donors and unseen diseases, targeting the transfer setting most relevant to translational research.
• Two tokenization families: TEDDY-G encodes genes by expression rank (Geneformer-style) while TEDDY-X uses binned expression values (scGPT-style), letting the authors compare how quantization affects downstream performance.
• Predictable scaling: Performance improves consistently with both training data volume and parameter count across the 70M, 160M, and 400M scales, following the scaling behavior observed in language models.
• Open weights: The TEDDY-G checkpoints are distributed on HuggingFace under Apache 2.0, a permissive license that allows commercial use.

#Technical Details

TEDDY uses a standard transformer backbone adapted to single-cell data, treating each cell's transcriptome as a set of gene tokens. Pretraining pairs a masked language modeling objective with an ontology-classification objective that predicts biological annotations, so supervision from curated labels shapes the learned embeddings during pretraining rather than only at fine-tuning. The models were trained on approximately 116 million cells drawn from public single-cell resources including CELLxGENE, spanning multiple tissues and disease conditions and both human and mouse. The family covers 70M, 160M, and 400M parameters. In the authors' evaluations, TEDDY delivers substantial improvements over prior single-cell models — including Geneformer, scGPT, scBERT, and Nicheformer — on disease-state identification and diseased/healthy cell classification for held-out donors, with gains scaling alongside data and model size.

#Applications

TEDDY is aimed at computational biologists and drug-discovery teams working with large scRNA-seq corpora who need to characterize disease states in new cohorts. Its cross-donor and cross-disease classification framing supports tasks such as labeling diseased versus healthy cells in patient samples, transferring disease signatures to unseen indications, and generating cell and gene embeddings for downstream analysis. Because it captures disease-related signal directly, it is well suited to target identification and mechanism-of-disease exploration where annotated reference data for a given condition are limited.

#Impact

TEDDY represents a pharmaceutical-industry entry into the single-cell foundation model landscape, and its annotation-aware pretraining offers a concrete alternative to the purely self-supervised objectives used by earlier models such as Geneformer and scGPT. By releasing the TEDDY-G checkpoints under a permissive Apache 2.0 license, Merck lowers the barrier for other groups to build on disease-focused single-cell representations. As a workshop preprint, its reported gains await broader peer review and independent benchmarking, and the released family currently covers the TEDDY-G (rank-based) checkpoints rather than the full set of TEDDY-X variants described in the paper.