PAGET

PAGET (Pathological image segmentation via AGgrEgated Teachers) segments the tumor microenvironment from routine H&E-stained histopathology slides, assigning every pixel to one of 13 tissue and cell classes in a single forward pass. It was developed by a group led by the University of Tokyo with pathology departments at Tottori University, the Japanese Red Cross Medical Center, Kanagawa Cancer Center, and National Cancer Center Japan, and published at MIDL 2026.

Two problems have kept comprehensive tumor microenvironment analysis out of reach. First, existing H&E segmentation models typically resolve only three to seven cell types, which is too coarse to characterize immune infiltration. Second, the ground truth they learn from is morphological — a pathologist's visual call on a nucleus — and that call is unreliable for cells with atypical appearance. Immunohistochemical restaining fixes the annotation problem by supplying protein-based labels, but it forces a separate model per antibody-cell type pair, which makes whole-slide analysis at scale computationally prohibitive.

PAGET resolves both at once by distilling an ensemble of restaining-trained specialists into one student. Its central design choice is that teacher predictions should be combined along the biological taxonomy of cell types rather than by flat voting: tissue context constrains major cell categories, which in turn constrain fine-grained leukocyte subtypes.

#Key Features

• Thirteen-class unified output: One model labels epithelium, stroma, and smooth muscle at tissue level, plus epithelial cells, fibroblasts, endothelial cells, red blood cells, lymphocytes, plasma cells, myeloid cells, eosinophils, neutrophils, and mitotic figures at nucleus level.
• Taxonomy-aware teacher aggregation: A three-level hierarchy — tissue context, then major cell categories, then leukocyte subtypes — lets confident fine-grained teachers override coarse classifications while preserving biological consistency.
• Heterogeneous teacher ensemble: SegPath models trained on immunohistochemical restaining supply tissue and leukocyte masks, dedicated MPO and ECP models separate neutrophils from eosinophils, MIDOG++ contributes mitotic figures, and HoverNet supplies nucleus instance boundaries.
• Two inference variants from one checkpoint: PAGET-S emits pixel-wise semantic labels directly; PAGET-H adds HoverNet instance masks and assigns each nucleus its majority class for panoptic output.
• Student exceeds its teachers: On external cohorts the distilled model consistently outperforms the full teacher pipeline it was trained from, an effect the authors attribute to regularization against distribution shift.

#Technical Details

The student is a SegFormer with a MiT-B5 encoder pretrained on ImageNet, taking 384x384 tiles at 20x magnification and predicting 14 classes: 13 tumor microenvironment components plus background. The arXiv preprint counts 14 components; the peer-reviewed version and the released code both specify 13 plus background. Distillation used 59,443 H&E tissue microarray images spanning 22 cancer types, yielding 8.7 billion labeled tissue pixels and 15.4 million predicted nuclei as pseudo-labels. Training ran for 48,000 iterations with AdamW at a learning rate of 6e-5, batch size 4, on eight NVIDIA H100 GPUs.

External validation covers PanopTILs (breast), Lizard (colorectal, with PanNuke excluded to avoid leakage through HoverNet), and KCCRC, a multi-institutional Japanese colon and gastric cohort. Tissue segmentation is scored by Dice and nucleus classification by per-class Matthews correlation coefficient. The ablation is the paper's clearest result: hierarchical aggregation beats flat aggregation by 24.5% relative on lymphocytes in PanopTILs, 25.7% on lymphocytes and 81.9% on eosinophils in KCCRC. On efficiency, PAGET-S processes a 384x384 tile in 4 ms on a single V100 — roughly 207x faster than the teacher ensemble and 301x faster than the full pipeline — covering a 100k x 100k pixel slide in about a minute against more than six hours for the teachers.

#Applications

PAGET is built for quantitative tumor microenvironment profiling at cohort scale: counting and spatially mapping immune infiltrates, stromal compartments, vasculature, and mitotic activity across large whole-slide image archives without per-dataset retraining. PAGET-S suits high-throughput screening where speed dominates; PAGET-H is the choice when nucleus boundaries matter or when stromal and immune identification must be precise. A board-certified pathologist reviewing output on adenoid cystic carcinoma, a cancer type absent from the 22 training types, found no obvious tumor cell misclassifications.

#Impact

PAGET widens the practical class vocabulary for H&E segmentation while removing the per-antibody model proliferation that immunohistochemical supervision had imposed, and it demonstrates that taxonomy-aware aggregation is a better way to fuse teachers with heterogeneous label spaces than majority voting. Its stated limitation is structural: because coarse levels constrain finer ones, errors in tissue or lineage prediction propagate downward, and granulocyte subtypes remain genuinely hard from H&E — 67 to 68% of high-probability granulocyte pixels also score high for myeloid. The implementation and distillation pipeline are released under the MIT license; trained student weights are not distributed, so reproduction requires running the teacher ensemble and retraining.