PathSegmentor

Segmentation is the workhorse of computational pathology — outlining tumour regions, glands, or individual nuclei is what turns a slide into measurable features. In practice it has been served by a patchwork of narrow models, each trained on one dataset with a handful of fixed classes, plus promptable general segmenters that require a human to click a point or drag a box for every object. Neither scales to the breadth of structures a pathologist actually names.

PathSegmentor replaces spatial prompting with language. A user types the name of the structure they want — "tumour epithelium", "lymphocyte", "nuclei" — and the model returns a mask, without any point, box, or scribble. Its authors describe it as the first text-prompted segmentation foundation model built specifically for pathology, and the accompanying PathSeg dataset is the resource that makes that possible: 275,000 image-mask-label triples aggregated and harmonized from 21 public sources, covering 160 semantic categories organized across 20 anatomical regions, three levels of histological structure (tissue, cell, nuclei), and 61 object types.

The model was developed by Zhixuan Chen, Hao Chen, and collaborators at the Hong Kong University of Science and Technology with the University of Science and Technology of China, the Chinese University of Hong Kong, and Tencent, and released as a preprint in June 2025. It sits alongside general biomedical segmenters such as MedSAM and BiomedParse, but is trained on pathology data at a category granularity those models do not reach.

#Key Features

• Natural-language prompting: Segmentation is driven entirely by a free-text label, removing the per-object spatial interaction that makes SAM-family models impractical on slides containing thousands of nuclei.
• 160-category open vocabulary: One checkpoint covers the full PathSeg label hierarchy, where specialized models typically handle around four classes each.
• Compact for its coverage: At 0.45B parameters the model is roughly a quarter the combined size of the 16 specialized SAM-Path models (1.86B) it replaces.
• Cross-dataset generalization: The model transfers to external nuclei and cell segmentation benchmarks it was not tuned on, rather than only to held-out splits of its own training distribution.
• Downstream interpretability: Segmentation outputs feed feature-importance estimation and imaging-biomarker discovery, letting diagnostic models be explained in terms of named histological structures.

#Technical Details

PathSegmentor pairs a FocalNet image encoder with a PubMedBERT text encoder, fusing the two streams in a joint feature-interaction module built from alternating cross-attention and self-attention layers; a transformer encoder-decoder with learnable queries then produces the masks. On the PathSeg test set it reaches an overall Dice of 0.671, against 0.502 for nnU-Net, 0.472 for SAM-Path, and 0.462 for DeepLabV3+ among specialized models. It exceeds SAM-Med2D by 0.239 Dice and MedSAM by 0.145 Dice among spatially prompted models, and improves on BiomedParse — the closest text-prompted comparator, at 0.242 — by 0.429 Dice. On five external datasets it records Dice scores of 0.705 on CPM17 nuclei, 0.485 on CPM15, 0.427 on Kumar, and 0.319 on Lizard, and surpasses BiomedParse by 0.074 Dice on CoNSeP colon cells. Code is released under an MIT licence; the trained weights are distributed through a Google Drive folder linked from the repository, and the processed PathSeg dataset release is contingent on redistribution permissions from its 21 constituent sources.

#Applications

The immediate use is quantitative slide analysis without per-task model training: delineating tumour and stroma for spatial statistics, counting and typing nuclei, or extracting morphological features to feed prognostic models. Because prompts are text, a pathologist or biologist can query structures directly rather than assembling an annotation set and training a segmenter, which lowers the barrier for exploratory work and for rare structures with too few examples to train on. The masks also serve as an explanation layer, grounding a diagnostic model's behaviour in named histological objects.

#Impact

PathSegmentor's substantive contribution is twofold: PathSeg consolidates a fragmented segmentation literature into one harmonized, hierarchically labelled corpus, and the model demonstrates that open-vocabulary text prompting — which has struggled to reach pathology-level granularity in general biomedical segmenters — is workable when the training data covers the vocabulary. The margin over BiomedParse is large enough to be a category difference rather than an increment. The caveats are the usual ones for a recent release: the work is a preprint, external Dice scores on the harder nuclei benchmarks (0.319 on Lizard) show that generalization is uneven across datasets, and the dataset's redistribution status leaves part of the reproduction path dependent on assembling the 21 sources independently.