SAM-Brain3D

SAM-Brain3D is a brain-specific 3D medical imaging foundation model for segmenting structures and lesions in volumetric brain MRI. Developed by researchers at the University of Cambridge and Shanghai AI Laboratory and introduced in May 2025, it addresses a persistent gap in neuroimaging AI: general-purpose segmentation models such as the Segment Anything Model (SAM) and its 3D medical successors are not tailored to the heterogeneous tissue contrasts and anatomy of brain MRI, while task-specific models fail to generalize across the many sub-modalities used in clinical and research neuroimaging.

The model was trained on over 66,000 brain image-label pairs spanning 14 MRI sub-modalities, learning brain-specific anatomical and lesion priors that transfer across downstream segmentation tasks. SAM-Brain3D is released alongside the Hypergraph Dynamic Adapter (HyDA), a lightweight module that adapts the frozen foundation model to new patients and modalities by fusing complementary multi-modal data and generating patient-specific convolutional kernels.

Together, the foundation model and adapter form a pipeline aimed not only at segmentation but at downstream brain disease analysis, including predicting Alzheimer's disease progression. The work was published in the journal Pattern Recognition (2025).

#Key Features

• Brain-specialized 3D foundation model: Pretrained specifically on volumetric brain MRI rather than general medical images, capturing neuroanatomy and lesion patterns that generic 3D segmenters miss.
• Broad modality coverage: Trained across 14 MRI sub-modalities, enabling transfer to the diverse contrasts (T1, T2, FLAIR, and others) encountered in neuroimaging studies.
• Hypergraph Dynamic Adapter (HyDA): A companion adapter that uses hypergraphs to fuse multi-modal imaging and non-imaging data and dynamically generates patient-specific kernels for multi-scale feature fusion.
• Disease analysis beyond segmentation: Demonstrated on clinically meaningful tasks such as progressive-vs-stable mild cognitive impairment (pMCI/sMCI) classification for Alzheimer's progression.
• Open implementation and weights: PyTorch code and a pretrained sam-brain3d checkpoint are publicly available for fine-tuning on downstream brain tasks.

#Technical Details

SAM-Brain3D builds on the promptable 3D segmentation paradigm of SAM-Med3D, adapting it for brain MRI with a volumetric vision-transformer backbone operating on 128x128x128 patches. Pretraining used more than 66,000 brain image-label pairs across 14 MRI sub-modalities. For downstream disease analysis, the Hypergraph Dynamic Adapter constructs a hypergraph over patients (default K=20 neighbors) to capture higher-order relationships among multi-modal features, then dynamically produces patient-specific convolutional kernels for multi-scale fusion. The authors evaluate the framework on the Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort, combining baseline and two-year follow-up MRI and PET imaging with non-imaging clinical features, and report improvements on brain disease analysis benchmarks over prior foundation-model and task-specific baselines.

#Applications

SAM-Brain3D targets neuroimaging researchers and clinical scientists who need accurate, generalizable segmentation across many MRI contrasts without training a bespoke model for each task. Its pretrained weights can be fine-tuned for structure and lesion delineation, while the HyDA adapter enables downstream prediction tasks such as forecasting conversion from mild cognitive impairment to Alzheimer's disease. Because it fuses imaging with non-imaging clinical variables, the pipeline is well suited to multi-modal brain disease cohorts where patient-specific adaptation matters.

#Impact

By extending the foundation-model approach explicitly to 3D brain MRI, SAM-Brain3D contributes a reusable backbone for the neuroimaging community, where labeled data are scarce and modality heterogeneity is high. The accompanying hypergraph adapter offers a general recipe for adapting frozen segmentation foundation models to new patients and modalities, and its peer-reviewed publication in Pattern Recognition signals validation beyond preprint. As a relatively recent release, broad community adoption and independent benchmarking are still emerging, and the public checkpoint and code lower the barrier for downstream brain disease research.