Histopathology feature extractor distilled from a 1.1B-parameter teacher into a ViT-Base, with leading robustness to staining and scanner variation.
No providers recorded yet. Browse all providers
H0-mini is a compact histopathology feature extractor produced by distilling a large pathology foundation model into a Vision Transformer roughly thirteen times smaller. It was built by Owkin and Bioptimus, and published at MICCAI 2025.
Pathology foundation models had been scaling along two axes at once, with checkpoints passing a billion parameters and pretraining corpora reaching millions of whole-slide images. That produced real gains and two side effects: embedding a large slide cohort became expensive, and robustness to the mundane variation of clinical practice — different scanners, different staining protocols — went largely unexamined even though it governs whether a model survives deployment. The teacher here is H-optimus-0, a 1.1 billion parameter ViT-giant; the student is a ViT-Base with 86 million parameters.
The notable result is not that distillation preserves accuracy but that it improves robustness. Across the PLISM benchmark of matched tissue imaged under varying stains and scanners, the distilled student retains feature correspondence far better than its own teacher and better than every other pathology foundation model evaluated. H0-mini has since served as the encoder backbone for downstream cell-level models including HistoPLUS.
Distillation follows the DINOv2 recipe with the teacher frozen. For two augmented views of each image, cross-entropy between teacher and student class-token prototype scores forms the DINO loss and the equivalent over patch tokens forms the iBOT loss, applied to all patches without masking. Stochastic depth and KoLeo regularization are removed, and an exponential moving average of the student becomes the final feature extractor. Training ran for 105,000 iterations at batch size 2,048 on 128 NVIDIA V100 32GB GPUs, totaling 4,350 GPU hours. The corpus is deliberately small: 43 million tiles at 224x224 and 20x magnification from 6,093 TCGA slides covering 16 cancer sites, matching Phikon's pretraining data exactly so the comparison isolates the effect of distillation.
On the HEST gene expression benchmark, H0-mini reaches a mean Pearson correlation of 0.404 across nine indications, third of the eleven extractors tabulated behind UNI2-h (0.429) and the teacher (0.422). On EVA it averages 0.782 across eight tasks, within 0.012 of the top entry but seventh in the table. A Wilcoxon signed-rank test over the 17 HEST and EVA tasks combined places it fourth overall. The original preprint's abstract advertised third place on HEST and fifth on EVA; the peer-reviewed version replaces those claims with a statement of comparable performance, and the tables are the authoritative figures. On PLISM — 46 tissue types under 13 staining conditions across 7 scanners, registered into 91 slides of 16,278 tiles each — it reports median cosine similarity of 0.92 cross-scanner and 0.79 across both axes, with top-10 accuracies of 0.86 and 0.18, the best in both columns.
H0-mini is a drop-in tile encoder for computational pathology pipelines: extract patch embeddings, aggregate them with attention-based multiple instance learning or mean pooling, and train a light head for biomarker prediction, subtype classification, survival modeling, or spatial gene expression. Its size suits cohorts running to thousands of slides or inference confined to one commodity GPU, and its robustness profile suits slides arriving from multiple sites with heterogeneous scanners and stains.
The work reframes distillation in digital pathology as more than a compression step, showing it confers an invariance property the teacher lacks, and the accompanying release of a processed PLISM benchmark made robustness a reported axis in subsequent pathology model releases. Two caveats attach. Distillation is not cheap — roughly 1.7x the compute of training an equivalent ViT-Base from scratch, since the teacher must be evaluated throughout — and the method requires access to the teacher's projection head, which is not always published. The weights carry a CC-BY-NC-ND-4.0 license behind a gated HuggingFace request, restricting use to non-commercial academic research, markedly more restrictive than the Apache-2.0 terms of the teacher.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.