MoPaDi

An attention heatmap tells a pathologist where a classifier looked, not what it saw there. A bright patch over a gland is equally consistent with the model reading the gland's architecture, the lymphocytes crowding around it, or nothing biological at all — just the pinker stain one scanning centre applies. The heatmap cannot separate those, so the reviewer supplies the explanation the picture does not contain.

MoPaDi (Morphing histoPathology Diffusion) answers with a counterfactual instead. A diffusion autoencoder encodes an H&E tile into a 512-dimensional semantic vector; that vector is pushed along the gradient direction a trained classifier defines for the opposite class; the diffusion decoder then renders a new tile from the shifted vector. Because the latent carries structural detail rather than a coarse summary, the counterfactual is the same tile with the decision-relevant morphology altered and everything else left alone. A pathologist reads the transition directly: goblet cells reappear, glands reorganize, mucin recedes. Those are the features the classifier is using.

The framework comes from the Else Kröner Fresenius Center for Digital Health at TU Dresden with clinical collaborators across Germany, Austria and the United States, first posted as a preprint in November 2024 and published in Cancer Research in 2026 under the title "Counterfactual Diffusion Models Provide Interpretable Explanations of Artificial Intelligence Models in Pathology." It is a generative counterpart to the same group's slide-level encoders COBRA and EAGLE, which compress a whole slide into a vector for prediction; MoPaDi instead synthesizes the image a prediction implies. Its distinguishing move is doing this under multiple-instance learning, so biomarkers labelled only at the patient level can be explained tile by tile.

#Key Features

• Classifier-guided counterfactual editing: Shifting the latent along a classifier's class-direction vector at increasing amplitude yields a morphing sequence rather than one edited image, so the reviewer sees which change arrives first.
• Weakly supervised explanations: A transformer-based MIL classifier supplies the manipulation direction for slide-level labels, extending counterfactuals to biomarkers never annotated on individual tiles.
• Stain–morphology decomposition: Prediction shifts split into style- and structure-related components per image, surfacing scanner or centre batch effects that a heatmap would pass off as biology.
• Pretrained autoencoders across four cohorts: Tissue-class, colorectal, breast and pancancer autoencoders ship with paired linear and MIL classifier heads, and the pancancer encoder serves both liver and lung subtyping without being refit.
• Unconditional tile synthesis: An optional latent DDPM fitted to the encoder's feature distribution samples new vectors, generating synthetic tiles from scratch.

#Technical Details

MoPaDi builds on the diffusion autoencoder formulation of Preechakul et al.: a DDPM decoder conditioned on a learnable convolutional semantic encoder, trained jointly under a noise prediction loss, with DDIM sampling at inference. Four autoencoders were trained — NCT-CRC-HE-100K (100,000 colour-normalized 224×224 px tiles, nine tissue classes), TCGA-CRC (588 slides from 582 patients, 512×512 px at 0.5 MPP), TCGA-BRCA (1,115 slides from 1,046 patients) and a TCGA pancancer set of 256×256 px tumour-region tiles spanning 32 cancer types. Training ran on 4–8 A100 GPUs until FID fell below 20. Reconstruction MS-SSIM ranged from 0.966 to 0.992, and FID between held-out and 10,000 synthetic tiles from 4.71 (pancancer) to 36.95 (NCT-CRC-HE-100K evaluated on CRC-VAL-HE-7K). Classifier heads reached AUC 0.98 ± 0.01 for colorectal tumour epithelium, 0.96 ± 0.01 for hepatocellular versus cholangiocarcinoma, 0.91 ± 0.01 for LUAD versus LUSC, 0.86 ± 0.02 for ILC versus IDC, and 0.73 ± 0.08 for MSI-high versus non-MSI-high. At manipulation amplitude 0.06, 98.8% of lung counterfactuals were predicted as the opposite class, and two pathologists in a blinded review rated 26.7%–63.3% of synthetic tiles as real.

#Applications

MoPaDi is an auditing and hypothesis-generation tool for groups that already train histopathology classifiers. Counterfactuals for an MSI-status model recovered mucinous differentiation, disordered glandular architecture and lymphocytic infiltration — features independently linked to microsatellite instability — while the same transitions exposed centre-specific colour shifts that the stain–morphology split then quantified. Cell-type segmentation showed immune-cell fractions moving as MSI biology predicts, a measurable check on plausibility. The unconditional sampler additionally produces synthetic tiles for augmentation or sharing.

#Impact

MoPaDi is, by the authors' account, the first generative model to produce counterfactual explanations inside a multiple-instance-learning framework, which matters because most clinically interesting pathology labels exist only at slide level. Its limits are real: analysis is tile-level, a new image domain still requires training its own diffusion autoencoder over days to weeks of multi-GPU time, generated tiles sometimes miss cues such as basement membrane or nuclear hyperchromatism, and cell-type generation carries consistent biases. A counterfactual explains one classifier's decision cues rather than making a diagnostic statement, and the stain-versus-morphology finding is reported for the MSI setting, not as a general result. Code is MIT-licensed; the trained checkpoints sit behind a HuggingFace access gate whose terms require a research or education declaration, non-commercial use only and no redistribution, even though the same repository's metadata declares GPL-3.0 — a conflict worth resolving with the authors before building on the weights.