PixCell

Computational pathology has been reshaped by discriminative foundation models — encoders such as UNI and Virchow that turn tissue tiles into embeddings for classification and retrieval. PixCell attacks the complementary problem: generating histology images rather than describing them. Several tasks in pathology are inherently generative — virtual staining, augmenting small annotated cohorts, and producing shareable surrogates for data that privacy rules keep inside an institution — and none of them are served by an encoder.

PixCell, from the computer vision lab at Stony Brook University with collaborators at Argonne National Laboratory, the University of Chicago, and the University of Utah, is a diffusion transformer trained on PanCan-30M, a corpus of 30.8 million tissue patches drawn from 69,184 H&E-stained whole-slide images spanning many cancer types and benign tissue. Its central design choice replaces the text captions that condition general-purpose image diffusion models with self-supervised embeddings from a pathology encoder. Because those embeddings are computed automatically from the slide itself, training scales to tens of millions of patches without a single human annotation.

Conditioning on real slide embeddings also makes generation controllable: the synthetic tissue inherits the morphology, stain characteristics, and site-specific appearance of whichever real data supplied the conditioning vector. Released in June 2025 with weights, training code, and synthetic datasets on HuggingFace, it is positioned as a general-purpose generative backbone for the field.

#Key Features

• Embedding-conditioned generation: Instead of text prompts, the model conditions on 1536-dimensional UNI-2h self-supervised embeddings, giving fine-grained control over the morphology of the generated tissue without any annotation.
• Progressive multi-resolution training: A three-stage schedule trains at 256x256, fine-tunes at 512x512, and finishes at 1024x1024, yielding two released checkpoints (PixCell-256 and PixCell-1024).
• Privacy-preserving surrogates: Institutions can condition on their own protected slides and share the resulting synthetic images in place of patient data, with a self-supervised encoder trained purely on 10 million synthetic TCGA patches reaching 76.64% balanced accuracy against 77.89% for real data.
• Virtual IHC staining: Retargeting the conditioning from H&E to immunohistochemistry on roughly paired tiles lets the model translate H&E inputs into multiple IHC stains, outperforming CycleGAN-family baselines on MIST and HER2Match.
• Targeted augmentation: Adding conditioned synthetic tiles to small training sets improves downstream classification F1 by roughly three points across several encoders.

#Technical Details

PixCell is a Diffusion Transformer adapted from the PixArt-Sigma framework, operating in the latent space of the Stable Diffusion 3 VAE. The released 1024-pixel model uses 28 transformer blocks with 16 attention heads and a 1152-dimensional hidden state, cross-attending to 16 conditioning tokens derived from the UNI-2h embedding of each 256-pixel sub-tile. PanCan-30M draws from TCGA diagnostic and fresh-frozen slides, CPTAC, GTEx, an internal Stony Brook cohort, and other public sources, all tiled at 20x magnification (0.5 microns per pixel). Training ran in three stages of 120,000, 60,000, and 80,000 iterations on up to 32 GPUs, using pre-extracted VAE features and encoder embeddings.

Image quality is scored with Frechet distances computed in pathology-encoder feature spaces as well as with Inception-based FID; PixCell-256 reports a Frechet H-optimus-1 distance roughly 50% below the next best generator on the held-out PanCan test split. Pathologists rated its outputs higher than alternatives across five quality criteria and assigned breast-cancer subtypes to synthetic regions with 94.4% agreement against matched real images. Weights are released under CC BY-NC-ND 4.0 and remain subject to the underlying UNI2 license; the synthetic datasets carry Apache 2.0 terms.

#Applications

The most immediate use is data augmentation for the many pathology tasks with a few hundred annotated slides, where conditioned synthetic tiles measurably improve classifier performance. The privacy pathway matters for multi-site collaborations: a hospital can release synthetic surrogates that preserve site-specific appearance without releasing patient slides. Virtual IHC staining targets a real cost center in clinical workflows, generating stain channels from an H&E section instead of consuming additional tissue and reagent time. Because the conditioning interface is a standard encoder embedding, downstream groups can also plug in ControlNet-style adapters for cell-level layout control.

#Impact

PixCell establishes generative modeling as a first-class capability in computational pathology alongside the discriminative encoders that have dominated the field, and its embedding-conditioning trick offers a template for scaling diffusion training in any imaging domain where captions do not exist. The open release of both checkpoints and multi-million-image synthetic datasets lowers the barrier for groups without slide-scale compute. Limitations are worth stating plainly: outputs inherit whatever biases exist in the scanners and cohorts of PanCan-30M, virtual IHC is validated retrospectively on roughly paired tiles rather than in clinical use, and the non-commercial, no-derivatives license constrains industrial adoption.