PathSegmentor

Segmentation is the workhorse of computational pathology — outlining tumour regions, glands, or individual nuclei is what turns a slide into measurable features. In practice it has been served by a patchwork of narrow models, each trained on one dataset with a handful of fixed classes, plus promptable general segmenters that require a human to click a point or drag a box for every object. Neither scales to the breadth of structures a pathologist actually names.

PathSegmentor replaces spatial prompting with language. A user types the name of the structure they want — "tumour epithelium", "lymphocyte", "nuclei" — and the model returns a mask, without any point, box, or scribble. Its authors describe it as the first text-prompted segmentation foundation model built specifically for pathology, and the accompanying PathSeg dataset is the resource that makes that possible: 275,200 image-mask-label triples aggregated and harmonized from 21 public sources, covering 160 semantic categories organized across 20 anatomical regions, three levels of histological structure (tissue, cell, nuclei), and 61 object types.

The model was developed by Zhixuan Chen, Junlin Hou, Hao Chen, and collaborators at the Hong Kong University of Science and Technology with the University of Science and Technology of China, the Chinese University of Hong Kong, Southern Medical University, and Tencent. It appeared as an arXiv preprint in June 2025 and was published as a Resource in Nature Computational Science in September 2026. It sits alongside general biomedical segmenters such as MedSAM and BiomedParse, but is trained on pathology data at a category granularity those models do not reach.

#Key Features

• Natural-language prompting: Segmentation is driven entirely by a free-text label, removing the per-object spatial interaction that makes SAM-family models impractical on slides containing thousands of nuclei.
• 160-category open vocabulary: One checkpoint covers the full PathSeg label hierarchy, where specialized models typically handle around four classes each.
• Robust to wording: Performance holds when the same structure is phrased differently, so prompts need not match a controlled vocabulary.
• Compact for its coverage: At 0.45B parameters the model is roughly a quarter the combined size of the 16 specialized SAM-Path models (1.86B) it replaces.
• Cross-cohort generalization: The model transfers to external public nuclei and cell benchmarks and to two hospital cohorts it was not tuned on, not only to held-out splits of its training distribution.

#Technical Details

PathSegmentor pairs a FocalNet image encoder with a PubMedBERT text encoder, fusing the two streams in a joint feature-interaction module built from alternating cross-attention and self-attention layers; a transformer encoder-decoder with learnable queries then produces the masks. Internal evaluation covers 45,018 image-mask pairs across the 16 PathSeg test datasets, with bootstrap confidence intervals and Holm-adjusted significance tests per dataset. Overall Dice reaches 0.671, against 0.502 for nnU-Net, 0.472 for SAM-Path, and 0.462 for DeepLabV3+ among specialized models. It exceeds SAM-Med2D by 0.239 Dice and MedSAM by 0.145 Dice among spatially prompted models, and improves on BiomedParse — the closest text-prompted comparator, at 0.242 — by 0.429 Dice. On five external datasets it records Dice scores of 0.705 on CPM17 nuclei, 0.485 on CPM15, 0.427 on Kumar, and 0.319 on Lizard, and surpasses BiomedParse by 0.074 Dice on CoNSeP colon cells. Code and analysis scripts are MIT licensed, with the journal version archived as v1.0.0 on Zenodo; the trained weights are distributed through a Google Drive folder linked from the repository, carrying no licence of their own.

#Applications

The immediate use is quantitative slide analysis without per-task model training: delineating tumour and stroma for spatial statistics, counting and typing nuclei, or extracting morphological features to feed prognostic models. Because prompts are text, a pathologist or biologist can query structures directly rather than assembling an annotation set and training a segmenter, which lowers the barrier for exploratory work and for rare structures. The masks also serve as an explanation layer: the authors use predicted structures to interrogate breast cancer classifiers through object-level perturbation and activation maps, grounding a classifier's behaviour in named histological objects.

#Impact

PathSegmentor's substantive contribution is twofold: PathSeg consolidates a fragmented segmentation literature into one harmonized, hierarchically labelled corpus, and the model demonstrates that open-vocabulary text prompting — which has struggled to reach pathology-level granularity in general biomedical segmenters — is workable when the training data covers the vocabulary. The margin over BiomedParse is a category difference, not an increment. Three limits temper that. External Dice on the harder nuclei benchmarks (0.319 on Lizard) shows generalization is uneven across datasets. The two hospital cohorts used for clinical validation are withheld at the contributing hospitals' request, so that evaluation cannot be reproduced independently. And PathSeg is redistributed only for the source datasets whose maintainers have granted permission, under a non-commercial licence, leaving the rest to be assembled from the originals under their own terms.