SynthFM

SynthFM is a promptable medical image segmentation model that was trained without a single real medical image. Developed at GE HealthCare with the University of Illinois Urbana-Champaign and presented at ISBI 2025, it pairs the frozen pretrained image encoder of Meta's Segment Anything Model with a mask decoder trained from scratch on a procedurally generated synthetic dataset, then applies the resulting checkpoint zero-shot to real clinical data.

The premise inverts the dominant strategy for adapting SAM to medicine. MedSAM and SAM-Med2D close SAM's domain gap by fine-tuning on very large corpora of expert-annotated real medical images — an approach that works but inherits the cost, licensing constraints, and coverage gaps of medical annotation. SynthFM instead asks whether the gap can be closed by writing down what makes medical images hard: low object-background contrast, modality-specific noise, blurred organ boundaries, and adjacent structures that share edges. Those properties are simulated analytically, and the decoder learns them from simulation alone.

Evaluated on 11 anatomical structures across nine publicly available CT, MRI, and ultrasound datasets, a single SynthFM checkpoint outperformed the zero-shot baselines it was compared against, with the largest margins in ultrasound — the modality where contrast is lowest and noise highest.

#Key Features

• Trained entirely on synthetic data: No real medical images, and therefore no expert annotations, are used at any stage of decoder training, sidestepping the annotation bottleneck that constrains supervised medical segmentation models.
• Shape-aware generation: Synthetic anatomy is built from Bézier curves with randomly sampled control points, superimposed on either a blank canvas or a simple circular phantom, producing the wide variety of organ silhouettes seen across patients and modalities.
• Boundary-aware generation: A second module derives multi-class label maps with 10–15 clusters, erodes a random subset of them, and assigns the eroded strip to background, creating the shared low-contrast boundaries between adjacent structures that make real segmentation hard.
• Modality-realistic noise and contrast: A noise library covering additive Gaussian, multiplicative Poisson, Perlin, speckle, and Rician noise, plus Gaussian blurring and a randomized contrast heuristic, is applied on the fly during training.
• Click-based prompting: Segmentation is driven by combinations of positive and negative clicks rather than boxes, with positive prompts placed near the target centroid and negative prompts sampled from a dilated ring around it.

#Technical Details

SynthFM keeps the SAM architecture intact: the image encoder is initialized from the original SAM weights and the mask decoder is trained from scratch with a Dice loss. Each epoch consists of 10,000 synthetic 1024×1024 images generated on the fly, with a batch size of 1, for 100 epochs at a learning rate of 1×10⁻⁴ under cosine decay. Evaluation covers eight abdominal structures in CT and MRI from AMOS, CHAOS, and TotalSegmentatorV2, and three ultrasound structures from CAMUS, HC, and FH-PS-AOP. Under one positive and two negative clicks, SynthFM reached 80.96 Dice on CT gallbladder against 56.16 for SAM, 63.40 for SAM 2, and 11.01 for UnSAM, and 77.11 on ultrasound left atrium against 35.98 for SAM. Averaged across organs at three positive and two negative clicks, it scored 83.48 on CT, 82.95 on MRI, and 81.55 on ultrasound. An ablation shows the two generation modules are complementary: the shape module alone nearly matches the full model on CT and MRI, while ultrasound degrades sharply without the boundary module. A separate comparison against MedSAM was run on ALFI, a label-free time-lapse microscopy dataset unseen by both models, where SynthFM scored 55.57 Dice on interphase cells to MedSAM's 12.35.

#Applications

The model suits radiology and ultrasound research groups that need a promptable segmentation backbone but lack the annotated corpora required to fine-tune one, and it is well positioned for modalities and anatomies with no public labeled data at all. Because the decoder never sees real images, it carries no patient data provenance or licensing encumbrance, which simplifies deployment in regulated settings. The demonstrated transfer to label-free microscopy suggests the same checkpoint is usable outside radiology.

#Impact

SynthFM establishes that analytically simulated data can substitute for annotated medical images when adapting a general-purpose segmentation foundation model, shifting the engineering effort from data collection to explicitly modeling the properties that distinguish medical images from natural ones. Its practical reach is currently limited: the head-to-head comparisons on the nine radiology datasets are against SAM, SAM 2, and UnSAM rather than supervised medical models, with MedSAM evaluated only on out-of-distribution microscopy, so the model's standing against real-data-trained medical segmentors on their home turf is not established. The model operates in 2D only, extension to 3D is left to future work, and no code or weights have been publicly released.