Medical image segmentation model that replaces MedSAM's manual box prompts with a diffusion prompt encoder and labels each mask by class.
No providers recorded yet. Browse all providers
MedSAM turned Meta's Segment Anything Model into a universal medical image segmenter, but it inherited SAM's interaction model: a human draws a bounding box, and the model returns a binary mask inside it. In a clinic that is two problems at once. Someone has to draw a box for every structure on every slice, and the mask that comes back is anonymous — a region of interest with no label saying whether it is liver, kidney, or tumour. Prompt quality also matters more than it looks; the segmentation is sensitive to where the box is placed.
AutoMedSAM, from researchers at Southwest Jiaotong University, Purdue University, and the University at Albany, removes the human from the prompting loop without discarding what MedSAM learned. It keeps MedSAM's image encoder and mask decoder and swaps the manual prompt encoder for a diffusion-based class prompt encoder: give it a class index, and it progressively denoises that index into the sparse and dense prompt embeddings the decoder expects. Because the prompt now carries a class identity, the output mask carries a semantic label too, turning MedSAM's semi-automated workflow into an end-to-end one that a non-specialist can drive.
The class prompt is projected through a linear layer to the prompt embedding dimension, then used to condition a forward diffusion process whose reverse pass produces the embeddings consumed by MedSAM's mask decoder. Evaluation spans four datasets and four modalities drawn from the CVPR 2024 Medical Image Segmentation on Laptop Challenge: AbdomenCT-1K (CT, abdominal organs), BraTS (MR-FLAIR, brain tumour), Kvasir-SEG (endoscopy, polyps), and Chest Xray Masks and Labels (X-ray, lung). Metrics are Dice Similarity Coefficient and Normalized Surface Distance at a 2-voxel tolerance. On AbdomenCT-1K, AutoMedSAM reports 94.58% DSC and 95.15% NSD against MedSAM's 93.51% and 92.97%, with the largest gain on the pancreas — the hardest of the five organs for every method compared. Training used a single NVIDIA RTX A40, batch size 16, 100 epochs, and AdamW at a 5e-4 learning rate. The repository currently carries the model and data-preprocessing code; training and inference code and the pretrained weights are described as forthcoming, and no license file has been added.
The intended setting is routine clinical imaging, where the bottleneck is not segmentation quality but the clinician time that interactive tools consume. A fully automatic, class-labelled segmenter fits radiotherapy contouring, organ volumetry, and lesion tracking, and it makes the model usable by technicians and researchers who are not trained to place prompts well. The same property helps in bulk annotation pipelines, where prompting thousands of slices by hand is the step that does not scale.
AutoMedSAM is one of a growing set of descendants that take MedSAM's pretrained representation and rework its interface, alongside SAM2-based successors that target volumetric propagation instead. Its specific argument — that a diffusion process is a natural way to synthesize prompt embeddings from a class label — is a reusable idea for any promptable segmenter that needs to run unattended. The caveats are that evaluation is confined to four benchmark datasets with a small set of segmentation targets, that class-index prompting presumes the classes of interest were seen in training, and that the paper is a preprint with weights not yet released.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.