Bidirectional generative model that renders a protein's fluorescence microscopy image from its sequence and cell context, and designs sequences back.
No providers recorded yet. Browse all providers
Most models that predict where a protein goes in a cell are trained on labels: a compartment name such as "nucleus" or "mitochondria", or a single number such as a saturation concentration. Labels are lossy: a protein forming nucleolar puncta and one spread evenly through the nucleoplasm can collapse into the same category, and every new question needs a new labelled training set. CELL-FM takes the opposite approach. It learns to regenerate the experimental readout itself, the fluorescence image of a protein in a cell, and leaves phenotyping to the image analysis pipelines already built for the real assay.
CELL-FM was developed by Dihan Zheng, Kibeom Hong and Bo Huang at UCSF, where Huang is also a Biohub San Francisco investigator, and posted as a bioRxiv preprint in September 2026. It is the latest model in the Huang lab's sequence-to-image line, after CELL-E, CELL-E 2 and CELL-Diff. It works in both directions. Given a protein sequence and images of a cell's nucleus and other landmarks, it samples plausible single-cell fluorescence images of that protein; given an image, it samples sequences consistent with it. Because the output is an image, one checkpoint serves many downstream questions without retraining.
The model is unrelated to CellFM, a single-cell transcriptomics foundation model from Sun Yat-sen University that shares only the name.
The protein channel and cell-context channels (nucleus, plus microtubules and ER for HPA) are encoded by a VAE and cut into patch tokens. The protein sequence is embedded with the 600M-parameter ESM C language model. A unified transformer encoder fuses image and sequence tokens, and three heads read from it: an MMDiT image generator trained with flow matching in VAE latent space, a sequence generator trained with an order-agnostic discrete-diffusion objective, and an auxiliary denoising masked-autoencoder image decoder. Stage 1 trains all three objectives with half of the image patches masked; stage 2 removes masking and fine-tunes the two generative objectives.
Pretraining uses the Human Protein Atlas subcellular resource, 12,894 proteins and about 1.1 million single-cell crops, with 1,200 proteins held out. On those unseen proteins CELL-FM reached lower distributional error than PUPS and CELL-Diff on both Inception-v3 FID and a microscopy-specific FID computed with a ViT trained on the same images. A variant fine-tuned on OpenCell (1,311 endogenously tagged proteins in live HEK293T cells, about 96,000 images) supports virtual staining. A third, trained on CondenSeq (14,598 intrinsically disordered 66-residue peptides imaged in U2OS nuclei, about 8 million images, 1,058 held out) adds expression-level conditioning.
Sliding a 25-residue window along the PRRSV nucleocapsid protein and scoring nuclear enrichment on the generated images recovered its two known nuclear localization signals. Shown nuclear-enriched or nuclear-depleted cells, the sequence head produced lysine- and arginine-rich NLS candidates and hydrophobic NES candidates. After AlphaFold 3 screening against importin KPNA1 and exportin CRM1, three generated NLSs matched SV40 NLS enrichment and one NES matched the PKIα NES when fused to mNeonGreen3K in U2OS cells. Virtual staining of all 1,311 OpenCell proteins on a shared nucleus yielded an embedding that separates nucleolar subcompartments and P-body proteins. On CondenSeq, virtual titration curves reproduced the effect of FG-repeat loss in NUP98 and of charge blockiness in DDX4 and LAF-1, and a 100-peptide substitution scan ranked aromatic residues, arginine and histidine as condensate drivers.
CELL-FM argues that a generative model of experimental readouts can replace many task-specific predictors, and demonstrates it with one pretrained model reused for localization prediction, motif design, representation learning and mutation scanning. Code is released under MIT on GitHub, and HPA, OpenCell and CondenSeq checkpoints are on Hugging Face with Colab notebooks for each application. The work is an unreviewed preprint. The wet-lab validation covers a small set of designed localization signals, and the condensate findings are in silico predictions. The authors note that HPA-trained outputs can reproduce immunofluorescence artifacts, and that combining datasets from different cell types and imaging conditions remains unsolved.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.