Lemur

Cell-type annotation is the bottleneck in single-cell analysis. Doing it by hand means an expert reading marker genes cluster by cluster, with nomenclature and classification criteria that differ between labs. Single-cell foundation models were supposed to automate this, but in practice models like Geneformer and scGPT still require a fine-tuning pass on every new dataset with labels supplied — which puts the expert back in the loop.

Lemur (Large Expression Model for Understanding scRNA-seq), from Baylor College of Medicine and the Jan and Dan Duncan Neurological Research Institute at Texas Children's Hospital with collaborators at Rice University and the Stevens Institute of Technology, gets around this by narrowing its scope and exploiting what that buys. It is built for Drosophila melanogaster alone, where whole-organism atlases already share a unified annotation schema. Because every training cell carries a label from one consistent ontology, Lemur can learn to emit annotations directly rather than learn embeddings that someone else must classify.

The architectural consequence is a hierarchical cell-type decoder. Given a cell, the decoder generates its label autoregressively — a broad category, then progressively finer terms, then a stop token — walking down the ontology tree the way an annotator would. Predictions are consistent across resolutions by construction, and no additional training is needed on a new dataset.

#Key Features

• Fine-tuning-free annotation: A pretrained checkpoint labels an unseen dataset directly, with no labelled examples or gradient updates required from the user.
• Hierarchical generative decoder: Cell types are produced as a token sequence from coarse to fine, e.g. CNS neuron then Kenyon cell then alpha/beta Kenyon cell, so coarse and fine predictions can never contradict each other.
• Confidence-thresholded output: Prediction confidence tracks accuracy, and a threshold of 0.75 retains 90% of cells while preserving every annotated cell type.
• Emergent batch correction: Encoder embeddings mix experimental batches while separating cell populations, despite no batch-correction objective during training.
• Cross-modality transfer: Trained on single-nucleus data, it annotates single-cell RNA-seq effectively, and holds up across wild-type and disease-model tissues.

#Technical Details

Lemur pairs a transcriptomic encoder with a hierarchical cell-type decoder. Input cells are reduced to their expressed genes, each gene's expression rank-binned within the cell, and gene and bin tokens embedded and summed, with a prepended <cell> token and a maximum length of 2,048 genes that covers 99.80% of transcriptomes without truncation. The encoder is 12 transformer layers at 512 dimensions with 8 attention heads; the decoder is 8 layers at the same width and uses the <cell> embedding as cross-attention memory. Training combines masked language modelling on gene tokens and expression values with next-token prediction over cell-type sequences, at a 10% masking rate, for 10 epochs on three A100 GPUs.

Pretraining used a whole-organism adult fly single-nucleus atlas of 624,458 cells and 16,195 genes, integrating the Fly Cell Atlas, Aging Fly Cell Atlas, and an optic lobe atlas into 219 cell types, organized as an ontology of 238 annotations with 199 leaf nodes and 19 top-level categories. On a held-out adult brain dataset Lemur identifies all 67 cell types present, with macro accuracy and F1 of 79% unfiltered. Across neurodegeneration models it reaches 83–86% accuracy on Alzheimer's (Tau and Aβ42), 84% on Parkinson's (alpha-synuclein), and 72–80% on Huntington's, at precision 0.88–0.93. On leg and wing integration it scores 0.57 batch correction and 0.83 biological conservation for an aggregate 0.73, above scANVI at 0.71, with the highest KMeans ARI (0.94) and NMI (0.88) against Harmony, scVI, and Scanorama.

#Applications

Lemur is aimed squarely at the fly community: annotating new tissue atlases, comparing disease models against wild type, and integrating datasets generated on different platforms without a separate batch-correction step. Because it requires no fine-tuning, it is usable by labs without machine-learning staff, and because Drosophila is genetically tractable with a short life cycle, computational predictions can be validated in vivo quickly — the iteration loop the authors argue is the real payoff of a species-specific model.

#Impact

Lemur makes a pointed case that curated, harmonized pretraining data can substitute for scale: a model trained on 624,458 cells from one organism achieves the fine-tuning-free annotation that much larger human models do not, because its labels come from a single consistent ontology. The generative hierarchical decoder is a reusable idea for any domain with a cell-type tree. Its limits are stated plainly by the authors: pretraining is exclusively Drosophila, so other organisms would need retraining, and novel cell types are mapped into the existing hierarchy rather than genuinely discovered. Weights are released on Hugging Face under the MIT license; the preprint has not been peer reviewed.