PRISM

Multi-sequence MRI is the workhorse of modern radiology precisely because it is heterogeneous: T1-weighted, T2-weighted, FLAIR, diffusion, and contrast-enhanced acquisitions each expose different tissue properties. That same heterogeneity is what breaks deep learning models. A network trained on one site's protocol tends to degrade when the echo time, field strength, or vendor changes, which is a central reason MRI models rarely survive the trip from a research cohort into clinical use.

PRISM is a foundation model built to absorb that variation rather than be defeated by it. Developed by Zelin Qiu, Hao Chen, and colleagues at the Hong Kong University of Science and Technology with radiology groups across mainland China and Hong Kong, it was pretrained with a paradigm that explicitly disentangles anatomically invariant structure from sequence-specific appearance while preserving high-level semantics. The preprint appeared in August 2025; the peer-reviewed version was published in Nature Biomedical Engineering, where the model is named MARS, and the released code and weights are distributed under that name.

Unlike most medical imaging foundation models, which specialize in a single organ or a single sequence type, PRISM was assembled from whole-body data and evaluated without per-task architectural redesign across five distinct task families. It is a different model entirely from the protein inverse-folding method that shares the PRISM name.

#Key Features

• Disentangled pretraining objective: The pretraining strategy separates anatomy-invariant features from sequence-specific acquisition variation, combining masked image reconstruction, cross-sequence image translation, anatomy-invariant contrastive learning, and prediction of acquisition metadata and body region.
• Whole-body, multi-organ coverage: Data spans ten anatomical structures including brain, heart, abdomen, prostate, and knee, rather than the single-organ scope of narrower MRI encoders such as BrainIAC.
• Volumetric 3D backbone: A Swin Transformer encoder with convolutional layers operates on full volumes, in the same architectural family as Swin-BOB but pretrained for general representation rather than a fixed segmentation label set.
• Five task families from one encoder: Diagnosis, segmentation, registration, progression prediction, and radiology report generation are all supported from the same pretrained weights.
• Open code and checkpoints: Source code is released under Apache-2.0 with pretrained weights available for download, alongside fine-tuning and pretraining scripts.

#Technical Details

The authors collected 64 datasets from public and private sources, of which 336,476 volumetric scans from 34 datasets — 8 public and 26 private — were curated into the pretraining corpus. The encoder is a 3D Swin Transformer arranged in a dual-branch disentanglement design, combining convolutional layers with transformer blocks and trained on all four self-supervised objectives simultaneously; the released checkpoint is designated MARS-336k after that corpus. Evaluation used a purpose-built benchmark of 44 downstream tasks assessed on 32 public datasets and 5 private cohorts. The preprint reported first-rank results on 39 of 44 benchmarks; the peer-reviewed version reports first place on 41 of 44, with statistically significant improvements over both randomly initialized baselines and prior medical imaging foundation models. Of the pretraining corpus, the 26 private datasets are available only on request to the corresponding authors under institutional data use agreements, so the full pretraining mixture cannot be reconstructed independently.

#Applications

The model targets the practical bottleneck in radiology AI: building a task-specific MRI model normally requires a large, protocol-matched annotated cohort. A pretrained encoder that already generalizes across sequences and organs lets a group fine-tune on far less labeled data for organ and lesion segmentation, disease classification, longitudinal progression prediction, inter-subject or inter-sequence registration, and draft report generation. Because the benchmark includes external cohorts acquired under unseen protocols, it is directly relevant to multi-site studies and to hospitals whose scanners differ from those used to develop a published method.

#Impact

PRISM addresses a gap between brain-specific MRI encoders and general medical-image models like MedSAM that treat MRI as one modality among many: it is pretrained at scale on MRI specifically, across the body, with sequence variation modeled explicitly rather than augmented away. Publication in Nature Biomedical Engineering alongside an Apache-2.0 code release and downloadable weights makes it one of the more reusable large-scale MRI encoders available. The main caveats are practical: most of the pretraining corpus is private, the model is released under one name and published under another, and clinical benefit remains to be demonstrated prospectively rather than on retrospective benchmarks.