Bidirectional masked transformer that predicts protein localization images from sequence and a nucleus image, and infills sequences from images.
No providers recorded yet. Browse all providers
Most protein localization predictors answer with a label: nucleus, cytoplasm, mitochondria. A label cannot say that a protein is concentrated at the nuclear rim, spread unevenly through the cytoplasm, or split between two compartments, and it cannot be run backwards to ask which sequence would send a protein somewhere specific. CELL-E 2 treats localization as an image instead. Given an amino acid sequence and an image of a cell's nuclei, it paints where the protein should appear in that cell. Given a nucleus image and a target localization pattern, it fills in masked stretches of sequence.
CELL-E 2 was developed by Emaad Khwaja, Yun S. Song, Aaron Agarunov and Bo Huang, with authors at UCSF, UC Berkeley, the Chan Zuckerberg Biohub San Francisco and Memorial Sloan Kettering Cancer Center. It was posted to bioRxiv in October 2023 and published at NeurIPS 2023. It succeeds CELL-E, an autoregressive decoder-only model from the same group that generated an image one patch at a time, could only map sequence to image, and was trained on the comparatively small OpenCell dataset.
The central change is a switch to a non-autoregressive, BERT-style masked encoder. Sequence tokens and image tokens sit in one input, random positions in both are masked during training, and the model learns to reconstruct them in parallel. Because either modality can be masked at inference, the same weights run in both directions. The same lab later took the sequence-to-image problem to diffusion in CELL-Diff.
Three tokenizers feed a single encoder-only transformer. A VQGAN compresses a 256 × 256 Hoechst nucleus image into 16 × 16 patch tokens from a 512-entry codebook; a second VQGAN of the same design tokenizes a protein threshold image, binarized at the median pixel value; and ESM-2 supplies per-residue sequence embeddings. Image tokens are masked on a cosine schedule with an expected rate of 64%, and sequence tokens on a linearly scaled version of that schedule capped at ESM-2's 15% masking rate. Loss is computed only on masked tokens. Four sizes were trained, matched to ESM-2 embedding widths of 480, 640, 1280 and 2560 with transformer depths of 68, 55, 25 and 5 layers, the depth limited by GPU memory.
Pretraining used 17,268 Human Protein Atlas proteins; OpenCell images of HEK293T cells were used for finetuning. Train and validation splits were made by clustering proteins at 50% sequence identity with PSI-CD-HIT, so that validation proteins are not close homologs of training proteins. Deeper, narrower models gave better images, and wider models gave better sequence infilling. Finetuning the HPA checkpoint on OpenCell, with its VQGAN image encoders also finetuned, gave the best OpenCell image predictions, but finetuning did not improve sequence reconstruction. Output resolution is limited to 256 × 256, and infilling works for scattered masked positions better than for long contiguous stretches.
CELL-E 2 lets cell biologists predict where an uncharacterized or engineered protein will go before cloning and imaging it, and its speed makes it practical to score many mutants in silico. The paper's design demonstration appended masked tokens to Green Fluorescent Protein and used the OpenCell 2560 model to fill in 255 candidate nuclear localization signals, then scored each with the best image model for predicted nuclear signal. Several high-scoring candidates were dissimilar to every entry in NLSdb, and DeepLoc 2.0 classified 89% of the GFP fusions as nuclear. These designs were evaluated computationally; the paper reports no wet-lab test of them.
CELL-E 2 showed that a single masked multimodal transformer can connect protein sequence and cell imaging in both directions, turning localization prediction into a route to sequence design. The authors released the code on GitHub under the MIT license, along with twelve checkpoints on Hugging Face across three training regimes and four sizes, and two hosted Hugging Face Spaces for image and sequence prediction. The group's later CELL-Diff model returns to the same sequence-image problem with diffusion.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.