BrainFM

BrainFM is a modality-agnostic, multi-task foundation model for human brain imaging that performs several core neuroimaging analyses from a single set of weights, regardless of how the input scan was acquired. Most medical-imaging models are trained for one modality and degrade sharply when faced with the uncalibrated, heterogeneous scans encountered in real clinical practice. BrainFM instead learns representations that are resilient to the appearance of the acquired image—its modality, contrast, deformation, resolution, and artifacts—so that the same network can be applied directly to CT and to T1-weighted, T2-weighted, and FLAIR MRI without modality-specific retraining.

The model was introduced in an August 2025 preprint by Peirong Liu and colleagues, a collaboration spanning Johns Hopkins University, the Athinoula A. Martinos Center for Biomedical Imaging (Massachusetts General Hospital and Harvard Medical School), the Danish Research Centre for Magnetic Resonance, and University College London. It extends the earlier Brain-ID line of work on contrast-agnostic anatomical representations toward a unified, multi-task system.

BrainFM's central idea is to learn a contrast- and modality-invariant feature space, then attach lightweight task heads on top. This lets one backbone cover synthesis, segmentation, geometry estimation, and registration, positioning it as a general-purpose preprocessing and analysis engine for brain images rather than a single-purpose tool.

#Key Features

• Modality-agnostic inference: A single model handles CT and T1w/T2w/FLAIR MRI without per-modality fine-tuning, remaining robust to differences in contrast, resolution, deformation, and artifacts.
• Five tasks in one network: Supports image synthesis (CT and T1w/T2w/FLAIR), anatomy segmentation, scalp-to-cortical distance estimation, bias field estimation, and image registration.
• Mild-to-severe intra-subject generation: A synthetic data engine generates a spectrum of degraded views of each subject, training the model to stay accurate as image quality declines.
• Real-synth mix-up training: Blends real and on-the-fly synthetic images during training, combining the diversity of synthesis with the realism of acquired scans.
• Open release: Code and pretrained weights are released under Apache-2.0, with demo scripts for inference and the data generator.

#Technical Details

BrainFM uses a five-level 3D U-Net backbone with 64 feature channels in its final layer, followed by a lightweight linear regression head appended per task and optimized end-to-end. Training draws on over 5,000 images from eleven public datasets—including ABIDE, ADHD200, ADNI3, AIBL, Buckner40, COBRE, ISBI2015, HCP, Chinese-HCP, MCIC, and OASIS3—augmented by the model's on-the-fly "mild-to-severe" generator and real-synth mix-up. On held-out evaluations the model reports strong T1w synthesis (PSNR ≈ 70.1, SSIM ≈ 0.97) and segmentation Dice scores around 0.85 for T1w MRI and 0.80 for CT, with competitive results across the other modalities. The reference implementation runs on Python 3.11 with PyTorch 2.0 and CUDA 12.2; pretrained checkpoints are distributed via OneDrive and a Hugging Face model repository.

#Applications

BrainFM targets neuroimaging researchers and clinical-imaging developers who need consistent analysis across the messy, multi-modal scans found in practice rather than the curated, single-modality data typical of benchmarks. Because one model synthesizes missing contrasts, segments anatomy, estimates geometry, and registers volumes, it can serve as a unified front-end to brain-imaging pipelines—harmonizing CT and MRI inputs, filling in absent modalities, and producing segmentations and registrations for downstream morphometric or longitudinal studies, including settings where only low-quality or uncalibrated scans are available.

#Impact

BrainFM advances the trend toward general-purpose, modality-agnostic foundation models in medical imaging, showing that a single contrast-invariant backbone can replace a collection of modality- and task-specific networks for brain analysis. Building on the Brain-ID lineage and released openly under Apache-2.0 with code and weights, it lowers the barrier for groups working with heterogeneous clinical scans. As a recent preprint its benchmarks await peer review and broader independent validation, and—like its contrast-agnostic predecessors—it is specialized for brain anatomy rather than arbitrary body regions or pathology detection.