MRI-CORE

MRI is the workhorse of soft-tissue imaging, but every new deep learning application built on it runs into the same wall: precise voxel-level annotation is expensive, radiologist time is scarce, and privacy rules make it hard to pool data across institutions. The practical consequence is that most clinical MRI segmentation projects start with a few dozen annotated slices, not a few thousand, and generic pretrained backbones transfer poorly from natural images.

MRI-CORE, developed at Duke University by the group of Maciej Mazurowski, is a general-purpose vision encoder pretrained specifically for this regime. It was trained self-supervised on Duke-110K, an in-house corpus of 6.9 million slices from 116,806 MRI volumes covering more than nine sequence types and over 18 body locations. The design goal is not zero-shot diagnosis but label efficiency: give a downstream task five annotated slices and the frozen-then-lightly-tuned MRI-CORE encoder should beat what a general segmentation foundation model or a from-scratch specialist can do.

Its closest comparators in the catalog are promptable segmentation models such as MedSAM, which MRI-CORE is benchmarked against directly, and MRI-specific pretrained encoders like Spark3D. MRI-CORE differs from both by covering the whole body rather than a single anatomy — notably excluding brain, face, and neck protocols, which were left out of the corpus for de-identification reasons.

#Key Features

• Pan-anatomical pretraining corpus: Duke-110K spans over 18 body locations and more than nine sequence types drawn from clinical exams collected between 2016 and 2020, rather than a single organ or a single protocol.
• SAM-initialized DINOv2 pretraining: Both student and teacher ViTs are initialized from the Segment Anything Model image encoder before DINOv2 self-distillation, a combination the ablations show beats DINOv2 from scratch and beats masked autoencoding from the same initialization.
• Five-slice segmentation regime: Downstream evaluation uses five annotated slices for fine-tuning and five for validation, matching how annotation budgets actually look in practice.
• Distribution-similarity analysis: The authors relate the size of MRI-CORE's improvement to the Fréchet Radiomic Distance between the pretraining corpus and each target dataset, giving a way to predict in advance where the model will help most.
• Zero-shot spatial structure: K-means clustering of the encoder's per-pixel features yields spatially coherent, semantically meaningful regions without any labels, where the same procedure on SAM or MedSAM features tracks low-level pixel similarity instead.

#Technical Details

The backbone is a ViT-Base image encoder — a deliberate compute-constrained choice the authors flag as a limitation — pretrained with DINOv2's student-teacher self-distillation plus masked image modeling. Ablations isolate the training recipe: sampling without replacement outperforms DINOv2's default sampling with replacement, volume-wise normalization beats slice-wise, and initializing both encoders from SAM beats initializing the student alone.

Few-shot segmentation was tested on five task families across ten datasets, both public and in-house: bone (Private-Bone, PIANO Hand MRI, SPIDER), muscle (Private-Muscle, MMThigh), breast tissue, organs (AMOS liver/pancreas/spleen, QIN & ISBI-MR prostate, HVSMR chamber and vessel), and tumor (LiverHccSeg). Against SAM, MRI-CORE raised 3D Dice by 1.75% to 17.4% across eleven of those tasks; against MedSAM by 3.29% to 27.3%; against nnU-Net by 1.2% to 33.34%. It trailed SAM on two tasks — SPIDER by 1.3% and HVSMR-Chamber by 0.7% — but led on 2D Dice, 3D Dice, and 3D normalized surface Dice when averaged over all datasets. Linear probing on a frozen encoder beat SAM and MedSAM on body location, sequence type, acquisition site, normal/abnormal status, gender, cirrhosis severity, and Pfirrmann disc grade.

#Applications

The intended user is a research or clinical group building an MRI analysis tool with a small annotation budget — segmenting a muscle group, an organ, or a lesion in a study cohort where only a handful of slices can be traced by hand. Beyond segmentation, the frozen encoder supports metadata classification tasks such as identifying sequence type or anatomical region, which is useful for triaging and auto-labeling heterogeneous imaging archives. The distribution-similarity analysis also gives teams a way to estimate expected benefit before committing to a fine-tuning run.

#Impact

MRI-CORE is one of the broader-scope MRI vision encoders released with open weights, and its ablation study is unusually explicit about which pretraining decisions actually mattered — a useful contribution given how rarely those choices are reported. Code is on GitHub and the checkpoint is distributed via Google Drive; the repository states Apache 2.0 terms in its README though it contains no license file. Two limitations bound its reach: the ViT-Base backbone leaves scaling headroom untested, and Duke-110K comes from a single institution and contains no brain, face, or neck MRI, so brain imaging in particular falls outside the pretraining distribution.