SAMPO-Path

Promptable segmentation models can outline almost anything in a natural image, but a histopathology slide breaks the assumption they rest on. Nuclei are small, densely packed, and heterogeneous, and a pathologist's click rarely means "segment this object" — it means "segment the tumor nuclei in this field and not the lymphocytes." That gap between a pixel-level training signal and clinical segmentation intent shows up as unstable, inconsistent masks whenever prompts are sparse or imprecise, which in practice they usually are.

SAMPO-Path, from the School of Biomedical Engineering and Technological Innovation at Fudan University, closes that gap with preference alignment rather than more pixel supervision. It fine-tunes the Segment Anything Model's ViT-B backbone with low-rank adapters under a Direct Preference Optimization objective, learning from ranked pairs of good and bad masks generated on the fly from prompts of varying quality. The authors present it as the first adaptation of DPO — a technique developed for aligning language models with human preference — to a pure vision foundation model.

The design decision that matters most for practical use is what the model is trained and tested on. Fine-tuning uses only two annotated histopathology datasets covering four tasks, but evaluation extends to 12 external datasets spanning different tissues, stains, and imaging modalities with no per-dataset retraining, so the reported transfer is genuinely zero-shot. The work is a preprint; no code repository or fine-tuned weights have been released, and no license beyond the arXiv distribution license is stated.

#Key Features

• Preference optimization for vision: A DPO-style objective ranks candidate masks by agreement with ground truth, teaching the model which segmentation a user actually wanted instead of only which pixels were positive.
• Online prompt-centric preference mining: Preference pairs are synthesized during training across deliberately degraded prompt sets, so robustness to sloppy or minimal prompting is learned rather than assumed.
• Multi-mask preference learning: SAM emits several mask hypotheses per prompt; SAMPO-Path treats that native ambiguity as supervision, building intra-prompt pairs between the best and worst hypothesis for fine-grained ranking.
• Hybrid loss for stable training: Preference optimization is combined with conventional pixel-level supervision applied to both preferred and dispreferred masks, which prevents the alignment term from destabilizing training.
• Parameter-efficient adaptation: Low-rank adapters at rank 64 leave the pretrained backbone intact, keeping the fine-tune cheap enough to run on a single GPU.

#Technical Details

The backbone is SAM's ViT-B image encoder initialized from the official checkpoint, adapted with LoRA modules at a default rank of 64 — a setting an ablation identifies as the best across all benchmarks — operating on 224x224 inputs. The objective sums a supervision loss and a preference term weighted by alpha = 1.0, with an inter- and intra-prompt balance of 1.0, and each sample is prompted with up to three positive and three negative points. Training runs for 10 epochs with AdamW at a learning rate of 3e-4 and weight decay of 0.01 on a single V100 GPU. Fine-tuning data comes from PanNuke, over 200,000 nuclei across 19 tissue types and five centers with five semantic categories, and CoNSeP, 24,319 nuclei from colorectal adenocarcinoma with four categories, supporting both a universal nuclei segmentation task and a category-specific one. On PanNuke universal segmentation with full training data the model reaches 70.36% IoU and 82.60% Dice, against 69.46% IoU for MedSAM. The margin widens sharply in the low-data regime: on category-specific segmentation with 10% of the training data it records 50.96% Dice versus 30.81% for MedSAM. Zero-shot results on held-out datasets include 82.13% Dice on TNBC, 81.14% on CPM17, and 90.75% on a fluorescence set, with comparisons against U-Net, SwinUNet, SAM-derived medical segmenters, and specialist nuclei methods.

#Applications

The target workflow is computational pathology annotation, where nuclei and gland segmentation underpins tasks from cell-type composition and tumor-infiltrating lymphocyte scoring to grading and spatial analysis. Because the model responds sensibly to a handful of imprecise clicks, it suits interactive loops in which a pathologist steers segmentation toward a specific cell population rather than accepting an exhaustive mask, and its zero-shot behavior across tissues and stains reduces the per-cohort annotation burden that usually blocks deployment on a new cohort.

#Impact

SAMPO-Path is notable less for a new architecture than for importing an alignment technique from language modeling into dense visual prediction, showing that intent, not just accuracy, is a trainable target. The low-data results are the strongest argument: a 20-point Dice gap over a strong medical SAM baseline at 10% training data is the regime that matters in pathology, where expert pixel annotation is the scarce resource. The limits are that the work remains an unreviewed preprint, that its capabilities are bounded by the upstream SAM checkpoint it adapts, and that neither the adapter weights nor the code have been published, so the reported zero-shot generalization cannot yet be independently reproduced.