MedLSAM

MedLSAM ("Localize and Segment Anything Model") addresses a persistent bottleneck in 3D medical image segmentation: the Segment Anything Model (SAM) and its medical variants are promptable, but they still require a human to supply bounding boxes or points on every slice of every scan. For volumetric CT, where a single study contains hundreds of slices and a dataset may contain thousands of studies, this prompting cost scales with the data and quickly becomes prohibitive. MedLSAM is presented by its authors as the first complete medical adaptation of SAM that removes this per-image burden.

The system's central contribution is MedLAM, a foundational localization model for 3D medical images, developed by researchers at Shanghai AI Laboratory, Shanghai Jiao Tong University, the University of Science and Technology of China, and West China Hospital of Sichuan University, and released as a preprint in June 2023 (published in Medical Image Analysis, 2025). MedLAM learns to locate any anatomical structure in a CT volume from only a handful of annotated template scans. By pairing MedLAM-generated bounding boxes with SAM (or MedSAM) for the actual mask prediction, MedLSAM segments arbitrary organs across an entire dataset while keeping the manual annotation effort fixed regardless of dataset size.

#Key Features

• Localize-anything foundation model: MedLAM identifies any anatomical region in a 3D CT volume given only a few template scans, acting as a reusable anatomical localizer rather than a single-organ detector.
• Constant annotation cost: The pipeline requires extreme-point annotations in three directions on just a few template images; this cost does not grow as the target dataset grows, unlike slice-by-slice prompting.
• Self-supervised pretraining: MedLAM is trained without segmentation labels using two self-supervision tasks — unified anatomical mapping (relative distance regression) and multi-scale similarity.
• Plug-in segmentation backend: MedLAM's predicted boxes drive SAM or MedSAM, so the framework benefits directly from improvements to the underlying promptable segmenter.
• Fully automated dataset segmentation: Once templates are annotated, the system segments target anatomy across all remaining scans with no further human input.

#Technical Details

MedLAM is trained on 14,012 CT scans using two self-supervised objectives: unified anatomical mapping (UAM), which regresses the relative 3D displacement between query and support patches to build a consistent anatomical coordinate frame, and multi-scale similarity (MSS), which refines localization through feature matching across scales. At inference, a few template scans with extreme-point annotations define each target structure, MedLAM predicts a 3D bounding box for that structure in unseen volumes, and the box is converted into prompts for SAM or MedSAM. The authors evaluate on two 3D datasets — StructSeg Task1 and WORD — covering 38 distinct organs. MedLAM's localization reaches accuracy comparable to fully supervised detection models, and MedLSAM matches the segmentation performance of SAM/MedSAM while eliminating the dataset-scaling annotation burden. Code is released under Apache-2.0, with public MedLAM checkpoints distributed alongside SAM ViT-B and MedSAM ViT-B weights.

#Applications

MedLSAM is aimed at researchers and clinicians who need to segment organs or structures across large CT cohorts — for example, radiotherapy planning, organ-at-risk delineation, automated dataset labeling, and large-scale anatomical quantification. Because MedLAM is a general localizer, it is reusable across segmentation targets and downstream segmenters, making it useful both as a standalone anatomical localization tool and as a front end that turns interactive promptable segmenters into automated batch-processing pipelines.

#Impact

By decoupling annotation effort from dataset size, MedLSAM offered an early and influential template for adapting the foundation-model "segment anything" paradigm to volumetric medical imaging, where naive per-slice prompting is impractical. The accompanying MedLAM localizer is of independent interest as a CT localization foundation model, and the project's public code and weights have made it a frequently cited reference point in the literature on SAM-for-medical-imaging. Its primary limitations are that MedLAM was trained and validated on CT (other modalities are not covered) and that final segmentation quality remains bounded by the underlying SAM/MedSAM backbone.