Loading…
Whole-slide pathology assistant that states the morphological findings behind each diagnosis, trained on 180k VQA pairs from 9,850 gigapixel slides.
No providers recorded yet. Browse all providers
Pathology assistants that answer questions about a slide tend to answer them badly in a specific way: they emit a bare label. Asked for a histologic grade, a model returns "Grade 2" with nothing behind it. A pathologist arrives at that grade from named observations — duct differentiation, nuclear pleomorphism, mitotic count — and a clinician cannot audit a conclusion whose supporting evidence was never stated. Existing whole-slide models were also trained and evaluated on benchmarks that never scored morphological description at all, so the gap went unmeasured.
WSI-LLaVA attacks the measurement problem and the model problem together. From Shenzhen University with collaborators at City University of Hong Kong, the University of Nottingham Ningbo China, Stanford University and the Hong Kong University of Science and Technology, the work first builds WSI-Bench, a morphology-aware benchmark of about 180k visual-question-answering pairs derived from 9,850 whole slide images covering 30 cancer types and 8,368 patients, then trains an assistant on it that reports its morphological findings before committing to a diagnosis. It was published at ICCV 2025.
The name sits in a crowded space: the catalog also holds LLaVA-Med, LLaVA-Rad and CXR-LLaVA, all LLaVA derivatives on different modalities. WSI-LLaVA is the gigapixel histopathology member of that family.
The slide encoder pairs Prov-GigaPath's DINOv2-pretrained tile encoder, applied to 256 x 256 patches, with a LongNet-based slide-level encoder; Bio_ClinicalBERT provides the text side during contrastive alignment. The released checkpoint is a 7B LLaVA-1.5 model fine-tuned with LoRA. WSI-Bench comprises 179,569 VQA pairs, split into 175,450 training pairs over 9,642 slides and 4,119 test pairs over 208 slides.
Averaged across the three capability groups, WSI-LLaVA scores 0.754 against 0.721 for Quilt-LLaVA on thumbnails, 0.590 for WSI-VQA and 0.507 for GPT-4o. On report generation it leads on every metric, reaching BLEU-1 of 0.480, ROUGE-L of 0.490, WSI-Precision of 0.380 and WSI-Relevance of 0.429. Zero-shot external validation holds up on three datasets outside TCGA, with 0.546 on WSI-VQA, an average of 0.553 on SlideBench-BCNB and 0.721 on CPTAC-NSCLC; the widest margins fall on the tasks needing detailed morphological interpretation, such as 46.3% versus 30.0% for histological grading. Removing the slide encoder costs roughly five points of overall average.
The intended use is a slide-level diagnostic assistant that a pathologist can interrogate: histological typing, grading, molecular subtyping and TNM staging, plus prognosis, treatment recommendation and draft report generation. Because each answer carries its morphological justification, the output is reviewable in the way a second-opinion note is, which is the precondition for clinical trust in this class of tool. WSI-Bench is independently useful as an evaluation harness for other whole-slide assistants.
The paper's central empirical claim — that morphological understanding correlates with diagnostic accuracy — reframes explainability as a driver of performance rather than a tax on it, and the WSI-Precision and WSI-Relevance metrics give the field a way to score that property. Both benchmark splits and model weights are public under Apache-2.0, though the checkpoint is access-gated and the published numbers correspond to a specific early checkpoint rather than the improved weights released later. Slide images themselves come from TCGA and must be downloaded separately, and the benchmark's reliance on TCGA reports means the distribution of cancer types and reporting conventions is inherited rather than designed.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.