Text-to-image transformer that paints where a protein sits in a cell as a probability map, from its amino acid sequence and a nucleus image.
No providers recorded yet. Browse all providers
Most computational predictors of protein localization answer a multiple-choice question: nucleus, cytoplasm, mitochondrion, plasma membrane. That framing breaks down for the many proteins that sit in several compartments at once, in varying proportions, and whose distribution shifts with cell type and cell-cycle stage. A class label cannot say that a kinase is mostly cytoplasmic in an interphase cell but crowds around the chromosomes during mitosis. CELL-E replaces the label with a picture: given an amino acid sequence and a fluorescence image of a cell's nucleus, it generates a two-dimensional probability map of where that protein is likely to be found in that particular cell.
CELL-E was developed by Emaad Khwaja (joint UC Berkeley–UCSF bioengineering program), Yun S. Song (UC Berkeley) and Bo Huang (UCSF); Song and Huang are also investigators at Chan Zuckerberg Biohub San Francisco. It was first posted to bioRxiv in May 2022, revised in March 2024, and published in the RECOMB 2024 proceedings (Research in Computational Molecular Biology, Lecture Notes in Computer Science).
The design borrows from DALL-E, OpenAI's autoregressive text-to-image model, with the protein sequence as the prompt. It began the Huang lab's sequence-to-image line, later extended by CELL-E 2 and CELL-Diff.
Training runs in three phases, as in DALL-E. First, a VQGAN learns to compress 256×256 nucleus images into a 16×16 grid of tokens drawn from a 512-entry codebook. Second, a VQGAN with the same architecture learns the same compression for binarized protein images, thresholded at each image's mean intensity. Third, a decoder-only transformer is trained on a single stream made up of 1,000 amino acid tokens, the nucleus-image tokens and the protein-image tokens, and learns their joint distribution. The released model has 32 transformer blocks with 16 attention heads and a fixed 768-dimensional TAPE embedding. At inference, rather than decoding one sampled token per patch, CELL-E weights every codebook entry by its predicted probability, which produces the continuous probability map.
The training data is OpenCell: 1,311 CRISPR-edited HEK293T cell lines, each with one endogenously tagged protein imaged by live-cell confocal microscopy alongside a nuclear stain. Z-stacks were collapsed to maximum-intensity projections and proteins split 80/20 into training and validation. TAPE outperformed UniRep, ESM-1b and one-hot embeddings. On validation proteins, nucleus proportion accuracy (how closely the predicted fraction of signal inside a Cellpose-segmented nucleus matches ground truth) was 0.81 ± 0.18. As a nuclear-versus-non-nuclear classifier it scored 0.72, against 0.79 for MuLoc and 0.69 for Subcons, both trained on localization labels. The transformer was trained on a single NVIDIA RTX 3090.
CELL-E suits cell biologists who want a spatial hypothesis for an uncharacterized protein before cloning a fluorescent fusion, and researchers studying localization signals. Fed the exact truncation mutants from a published deletion series for DNA topoisomerase I, it largely reproduced the observed localizations, including the finding that residues 1–67 are not sufficient for nuclear import despite containing a putative signal. The attention analysis on the same protein recovered the putative signals in the experimentally verified 148–199 range and also picked up part of the residues 117–146 nuclear localization signal reported by Mo et al., alongside residues 89–107 as a further candidate. This suggests the model can help prioritize constructs for wet-lab testing.
Its authors described CELL-E as the first demonstration of text-to-image synthesis in cell imaging. It reframed localization prediction as conditional image generation and made the reference cell part of the question. Its limits are the ones the authors name: OpenCell covers about 1,300 proteins in one cell line, and with only a nuclear stain as reference, cytoplasmic predictions are less constrained. Those limits shaped the follow-ups: CELL-E 2 trained on the larger Human Protein Atlas and added image-to-sequence generation, and CELL-Diff replaced autoregressive generation with diffusion. The code is MIT-licensed and a trained checkpoint is on HuggingFace.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.