vesselFM

A capillary bed in a two-photon microscopy stack and a circle of Willis in a contrast-enhanced MR angiogram are the same anatomical object, but as images they share almost nothing: the vessels differ by orders of magnitude in calibre, the artifacts and signal-to-noise ratios are modality-specific, and the surrounding tissue looks entirely different. A segmentation network trained on one collapses on the other, so every new imaging protocol has demanded its own round of voxel-level tracing — a punishing annotation task, because a vessel tree is thin, branching, and wrong everywhere at once when it is wrong.

vesselFM attacks this by widening the image distribution rather than the label taxonomy. It is trained in a plain supervised fashion on image–mask pairs drawn from three heterogeneous sources: D_real, a curated corpus of real annotated vascular volumes; D_drand, semi-randomized volumes whose foreground geometry comes from vascular corrosion casts pasted onto procedurally textured backgrounds; and D_flow, samples from a mask- and class-conditioned flow-matching generative model. The synthetic halves deliberately overshoot the appearance space of any real dataset, leaving the network with no strong prior on what a vessel image should look like — only on what a vessel shape is.

The model comes from Bastian Wittmann, Yannick Wattenberg, Tamaz Amiranashvili, Suprosanna Shit and Bjoern Menze at the University of Zurich, with co-authors at ETH Zurich and the Technical University of Munich, and was presented at CVPR 2025. It is positioned against general-purpose 3D medical segmentation foundation models such as SAM-Med3D, VISTA3D and Medical SAM 2, whose organ-class and point-prompt framing transfers poorly to fully-connected tubular structure.

#Key Features

• Zero-shot transfer to unseen modalities: the shipped checkpoint segments volumes from imaging domains held out of training entirely, with no fine-tuning and no prompt.
• Three complementary data sources: real curated data supplies fidelity, domain randomization supplies appearance breadth, and the flow-matching generator fills the space between them with anatomically coherent samples.
• Mask- and class-conditioned generation: the flow-matching model is conditioned on synthetic masks rather than real ones, so generated images inherit vessel topology free of annotator-specific bias.
• One- and few-shot adaptation: the checkpoint fine-tunes on a single 128³ annotated patch, or three, when a domain needs more than out-of-the-box accuracy.
• Topology-aware scoring: centerline Dice (clDice) is reported alongside Dice, so vessel-tree connectivity is measured rather than voxel overlap alone.

#Technical Details

The segmentation network is a plain 3D U-Net — MONAI's reimplementation of the nnU-Net architecture — operating on 128³ patches. D_real comprises more than 115,000 such patches curated from 17 annotated sources, split into 23 dataset classes by tissue, modality and protocol, covering brain, liver and kidney vasculature in human, mouse and rat. The domain-randomization pipeline draws foreground geometry from 1,137 vascular patches of shape 250³ and emits 500,000 image–mask pairs; the flow-matching model, sampled by Euler integration over 100 steps, contributed 10,000 pairs generated on a single RTX A6000 in three days. Training samples the sources at roughly 70% D_drand, 20% D_real and 10% D_flow.

On four evaluation datasets excluded from training, the base checkpoint reaches zero-shot Dice of 74.66 on SMILE-UHURA human brain MRA, 67.49 on BvEM mouse brain volume electron microscopy, 46.94 on mouse brain OCTA and 29.69 on MSD8 hepatic CT — ahead of VISTA3D on MSD8 by 5.86 Dice despite VISTA3D having trained on 11,454 CT volumes including MSD8 itself. Ablating the synthetic sources drops SMILE-UHURA zero-shot Dice to 65.45 and clDice by 11.74. Transformer backbones do not help under the same recipe: SwinUNETR-V2 scores 74.54 and UNETR 46.74.

#Applications

The immediate use is segmenting vasculature in a dataset with no annotations at all — a light-sheet whole-brain volume, an OCTA acquisition, a new CTA protocol — producing a starting mask that a researcher corrects rather than draws. That covers vascular morphometry in preclinical microscopy, cerebrovascular analysis in stroke and aneurysm imaging, and hepatic vessel delineation for surgical planning. The inference script pulls the checkpoint automatically and takes NIfTI volumes, so it slots into an existing pipeline without a training stage.

#Impact

vesselFM demonstrates that for a task with strong shape priors and weak appearance priors, synthesizing the appearance variation beats collecting more of it — a recipe that extends beyond vasculature to other thin, branching structures. The release is split-licensed: code is GPL-3.0, while the checkpoints carry an Open RAIL++-M license restricting them to research and non-commercial use, and downstream use must also respect the licenses of the 17 constituent datasets. D_real is not redistributed as a corpus; the repository documents per-dataset access instead, and several sources require an explicit request. One caveat matters for reproduction: checkpoints released after publication were trained on a broader set that includes the paper's evaluation data, so the zero-shot figures above correspond to the base checkpoint only.