PathoSAM

Counting and delineating nuclei is one of the most common quantitative tasks in digital pathology, and one of the most brittle. A network trained on colorectal H&E sections degrades on breast tissue, on immunohistochemistry, on cryosections, on a different scanner — which in practice means every new cohort begins with a round of manual annotation and retraining. Vision foundation models offer an escape: the Segment Anything Model (SAM) generalizes across natural image domains and accepts point and box prompts, so a user can correct rather than retrain.

PathoSAM, from the Computational Cell Analytics group at the University of Göttingen, adapts that idea to histopathology. It fine-tunes SAM on a diverse collection of openly available annotated histopathology images, producing generalist checkpoints for automatic and interactive nucleus instance segmentation that transfer to tissue types and stains they were never trained on. The work extends the same group's MedicoSAM and micro-sam line of domain-specialized Segment Anything variants, and was published at MIDL 2025.

The practical claim is not that a foundation model beats every specialist — it is that one model handles the long tail of datasets for which no specialist exists, and that when it falls short, a few prompts close the gap faster than annotating a new training set.

#Key Features

• Automatic instance segmentation: Nuclei are segmented without any user input, reaching state-of-the-art results for automatic nucleus instance segmentation in histopathology.
• Interactive correction: Point and box prompts refine masks iteratively, which matters most on the hardest domains — on cryosectioned tissue the model improves drastically through iterative correction where automatic segmentation struggles.
• Whole-slide inference: The package ships a command-line tool that tiles and segments an entire whole-slide image, plus example data, rather than leaving WSI handling to the user.
• Semantic segmentation head: A convolutional decoder attached to the fine-tuned image encoder extends the model to semantic nucleus classification across tissue classes.
• Annotation-tool integration: The checkpoints are compatible with the group's existing annotation tooling, so the model can be dropped into a human-in-the-loop labeling workflow.

#Technical Details

The authors assemble 14 publicly available histopathology datasets. Six H&E collections with nucleus instance annotations — including MoNuSeg, Lizard, PanNuke, PUMA, TNBC, and the CPM sets — are used to train the generalist models, each with a reserved test split; the remaining eight are held out entirely to measure out-of-distribution behavior. Three generalists are trained, one for each SAM image encoder size (ViT-B, ViT-L, and ViT-H), with ViT-L scoring highest overall, and an additional ViT-B model is trained on PanNuke alone to isolate the benefit of multi-dataset training — the generalist matches or beats it everywhere except an immunohistochemistry tissue microarray set. For automatic segmentation the model is compared against five established methods including HoVerNet; for semantic segmentation on PanNuke's five nucleus classes it outperforms popular baselines but does not surpass CellViT, which remains the state of the art on that task. Code and checkpoints are released under an MIT license.

#Applications

PathoSAM suits any lab that needs nucleus counts, densities, or morphometrics from histology but does not have annotated data for its particular tissue, stain, or scanner — tumor microenvironment quantification, nuclear atypia scoring, and cell-density mapping across a slide. Because it is prompt-driven, it also functions as an annotation accelerator: a pathologist prompts and corrects, producing ground truth far faster than drawing outlines, whether for direct analysis or to bootstrap a specialist model. The whole-slide command-line entry point means the common case requires no code.

#Impact

PathoSAM fills a gap that had been noted repeatedly — earlier SAM adaptations to pathology were partial, addressing one task or one dataset — with a model covering automatic and interactive instance segmentation plus a semantic extension, all openly licensed. Coming from the group behind micro-sam, it inherits an established tooling ecosystem, which matters more for adoption than benchmark deltas. The candid framing is a strength: the authors report where PathoSAM does not win, including semantic segmentation against CellViT and automatic segmentation on cryosectioned and immunohistochemistry-stained tissue, both domains underrepresented in the H&E-dominated training mix.