PathOrchestra

Pathology foundation models are now numerous, but the gap between a strong benchmark result and a model a hospital can actually deploy is wide. Most are validated on a handful of public cohorts — TCGA subtyping, CAMELYON metastasis detection — which leaves open whether the same frozen features hold up across the messier variety of real diagnostic work: slide quality control, immunohistochemistry scoring, rare lymphoma subtypes, and the many organ systems public datasets underrepresent.

PathOrchestra, from Shanghai AI Laboratory with Fourth Military Medical University, Tsinghua University, and collaborators across Chinese and Hong Kong institutions, is a self-supervised whole-slide-image encoder built specifically to be tested at that breadth. It was pretrained on roughly 300,000 H&E-stained whole-slide images covering 20 tissue and organ types from multiple centers, then evaluated on 112 downstream clinical tasks assembled from 61 private and 51 public datasets — a validation surface several times larger than is typical for the class.

The evaluation protocol is deliberately plain. The backbone is used as a fixed feature extractor, with only lightweight task heads on top and no per-task architectural changes, which makes the results a direct statement about how general the learned representation is rather than about task-specific engineering.

#Key Features

• Breadth-first clinical validation: The 112 tasks span slide preprocessing and quality control, pan-cancer classification, lesion identification, multi-cancer subtyping, biomarker assessment, spatial gene expression prediction, and structured report generation.
• Structured report generation: The model produces patient-level structured reports for colorectal cancer and for lymphoma, combining H&E subtype prediction with qualitative assessment of 29 immunohistochemistry markers — an output format most pathology foundation models do not attempt.
• Frozen-backbone transfer: Every downstream result comes from the same pretrained encoder without task-specific retraining, so a new task requires only training a small head on extracted features.
• Pre-extracted embeddings: Slide-level and tile-level feature embeddings are distributed alongside the weights, letting groups without GPU capacity for inference over full slides run downstream experiments directly.

#Technical Details

The backbone is a Vision Transformer with roughly 304 million parameters, operating on 256×256-pixel tiles at 20× magnification. Pretraining follows the DINOv2 recipe — a teacher-student pair updated by exponential moving average, combining the DINO objective on global crops, the iBOT masked-patch objective, and KoLeo regularization — sampling an average of 500 non-overlapping tiles per slide from a 262.5 TB corpus.

Downstream evaluation covers 27,755 whole-slide images and 9,415,729 regions of interest, with accuracy above 0.950 on 47 of the 112 tasks. Reported results include 0.988 AUC and 0.879 accuracy on a 17-class in-house pan-cancer classification task, 0.964 AUC on 32-class TCGA FFPE pan-cancer classification, 0.954 accuracy on bladder cancer screening, and 0.975 accuracy with 0.994 AUC separating lymphoma from reactive hyperplasia. On the HEST benchmark for predicting spatial gene expression from histology, the model averaged 1% above UNI and 5.7% above Prov-GigaPath across five cancer types (LUAD, READ, SKCM, PRAD, and IDC).

#Applications

The immediate use is as a drop-in feature extractor for computational pathology pipelines: a group with a labeled slide cohort trains a lightweight classifier on PathOrchestra embeddings rather than fine-tuning an encoder from scratch. The task coverage maps onto concrete diagnostic workflows — screening slides for tumor presence, assigning cancer subtype, scoring immunohistochemistry markers that would otherwise require a separate stain read, and drafting the structured findings a pathologist then reviews. The released embedding set also makes the model usable as a benchmarking baseline without running slide-scale inference.

#Impact

PathOrchestra's argument is about validation discipline as much as modeling: it shows that a single self-supervised encoder, held fixed, reaches clinically useful accuracy across a task set an order of magnitude broader than the field's usual benchmark suite, and it extends into structured reporting for colorectal cancer and lymphoma, which foundation models rarely address. The work is peer reviewed, published in npj Digital Medicine in November 2025 under a CC BY-NC-ND 4.0 license. Access terms are narrow in every direction. The GitHub repository carries an Apache-2.0 license, but it ships documentation rather than a codebase: the train/, eval/, and Downstream Tasks/ directories each hold a single README — a pretraining hyperparameter table, a pointer to an external evaluation library, and a table of dataset links — leaving the feature-extraction snippet in the top-level README as the only executable code. The pretrained weights carry CC BY-NC-ND 4.0 restricted to academic research and are gated behind manual approval requiring an institutional email address, ruling out commercial use and redistribution of derivatives. The 61 private datasets that make up the majority of the evaluation are not released, so much of the headline validation cannot be independently reproduced.