Loading…
Cross-modal translator turning routine H&E slides into virtual 21-channel multiplex immunofluorescence maps of the tumor microenvironment.
No providers recorded yet. Browse all providers
A PD-L1 immunohistochemistry stain answers one question about one protein on one tissue section. Multiplex immunofluorescence (mIF) answers twenty-one at once and keeps every answer in place on the slide, the reason it anchors tumor microenvironment work. It also consumes reagents, a hyperplexing scanner, and a tissue section that is often the last available — so mIF cohorts number in the dozens while hematoxylin and eosin (H&E) slides fill hospital archives by the million. GigaTIME holds that the morphology already visible in an H&E slide carries much of the cell-state information mIF measures, and learns that mapping directly: given an H&E tile, it labels every pixel activated or not for each of 21 protein channels, producing a registered virtual mIF image.
The model was published in Cell in December 2025 by Microsoft Research with Providence and the University of Washington, the collaboration behind the GigaPath whole-slide foundation model. Its translator was trained on roughly 40 million cells from paired H&E and mIF slides of the same tissue, a corpus generated in-house for want of a public equivalent.
What that fixed checkpoint enabled is the paper's headline: applied unchanged to 14,256 patients from 51 Providence Health hospitals and over a thousand clinics, it produced 299,376 virtual mIF whole-slide images spanning 24 cancer types and 306 subtypes, plus 214,200 more for 10,200 tumors from The Cancer Genome Atlas. GigaTIME is the convolutional predecessor of GigaTIME-Flash, which keeps this task and corpus but swaps the CNN backbone for a distilled vision transformer.
GigaTIME is a patch-based encoder-decoder built on NestedUNet (UNet++), a convolutional U-Net variant with dense nested skip connections, roughly 9 million parameters. A 256x256 H&E tile maps to same-resolution output maps — 21 protein channels plus two background channels — each pixel a binary activation call, trained under a combined binary cross-entropy and Dice loss for 300 epochs. Training data came from 21 matched slide pairs from 21 lung adenocarcinoma patients at Providence, imaged on the Lunaphore COMET platform to yield 441 mIF images and, after registration and segmentation, roughly 40 million labeled cells.
Evaluation ran at three scales. At pixel level, held-out Dice reached 0.72 for DAPI against 0.12 for an average-activation baseline. At cell scale, counting activated pixels in 8x8 windows, Pearson correlation was 0.59 for DAPI where CycleGAN managed 0.03. At slide scale, patch-wise activation ratios matched measured mIF at Spearman 0.98 for DAPI and 0.56 across all channels. Accuracy is stratified by biology: nuclear markers such as DAPI and Ki67 predict best, diffuse cytoplasmic and surface markers worst. On breast and brain tissue microarrays — cancer types absent from training, and physically unlike whole slides — performance dropped but the channel ranking held.
The intended use is retrospective research at a scale mIF cannot reach: profiling immune states across an archive of diagnostic H&E slides. In the paper this surfaced 1,234 significant associations between the 21 virtual channels and 20 clinical biomarkers — high tumor mutational burden tracking with CD138, CD20, CD68 and CD4; KRAS mutations tracking negatively with CD3 and CD8, an immune-excluded pattern; reported PD-L1 status agreeing with the virtual PD-L1 channel. Activations also stratified patients by stage and survival, the combined signature beating any single channel.
GigaTIME established that a translator trained once on paired data can stand in for a scarce assay across a whole health system, making population-scale microenvironment analysis possible. Concordance between the Providence and TCGA virtual populations was high (Spearman 0.88 across subtypes), while Providence yielded 33% more significant associations — an argument for real-world cohorts over curated ones. The limits are plain: the paired corpus is 21 lung adenocarcinoma patients at one health system, correlations for surface and cytoplasmic markers are moderate, associations are statistical rather than causal, and every result is in-silico. Microsoft licenses the weights permissively but restricts them to research, excluding clinical care and any deployed use.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.