MAISI

A diagnostic chest-abdomen CT runs to roughly 512 × 512 × 768 voxels — more than a 3D generative network can hold in a single accelerator's memory. Earlier medical image generators therefore worked at 128³ and upsampled, or built the volume slice by slice, which yields slices that look convincing alone and disagree along the through-plane axis. MAISI (Medical AI for Synthetic Imaging) generates the whole volume at diagnostic resolution in one unified 3D framework and emits a voxel-level anatomical segmentation alongside it, so the synthetic scan arrives already annotated.

It gets there by moving generation into a compressed latent space and splitting the arithmetic that remains. A VAE-GAN volume compression network encodes a CT volume into a much smaller latent grid, a latent diffusion model conditioned on body region and voxel spacing produces new latents there, and the decoder returns them to voxel space. Because that grid carries flexible spatial dimensions, output size and voxel spacing become inference-time arguments rather than properties baked in at training. Tensor splitting parallelism then partitions the remaining large 3D feature maps into overlapping segments across devices and stitches them back, avoiding the seams sliding-window inference leaves when the network output is image intensity rather than a smooth probability map.

MAISI was developed by NVIDIA with collaborators at the National Institutes of Health and the University of Arkansas for Medical Sciences, and published at WACV 2025.

#Key Features

• Paired images and 127-class masks: A ControlNet conditions the frozen diffusion model on a segmentation mask spanning 127 anatomical structures, so each volume ships with a label map usable as segmentation ground truth.
• Geometry chosen per request: Output dimensions (256, 384 or 512 in x and y, up to 768 in z) and voxel spacing (0.5–5.0 mm) are inference parameters, so one checkpoint can match the acquisition settings of the cohort it augments.
• Controllable anatomy size: Ten organ and lesion classes — liver, gallbladder, stomach, pancreas, colon, lung tumor, bone lesion, hepatic tumor, colon cancer primaries and pancreatic tumor — take a size percentile from 0 to 1, or −1 to omit the structure.
• Tumor inpainting: A second ControlNet inserts synthetic lesions into otherwise healthy anatomy, augmenting tumor cohorts without first generating a plausible whole-body layout.
• Frozen generative backbone: Adapting to a new task trains only a ControlNet; the compression network and diffusion model stay fixed, so a new lesion class costs a conditioning branch rather than a retraining run.

#Technical Details

The compression network is a 3D variational autoencoder trained against a discriminator on 39,206 CT and 18,827 MRI volumes, combining L1 reconstruction, LPIPS perceptual, KL regularization and adversarial losses. The latent diffusion model is a time-conditional 3D U-Net trained on 10,277 CT volumes from multiple public datasets covering head-neck, chest, abdomen and lower-body regions, conditioned on top and bottom body-region indices plus a three-vector of voxel spacings. Implementation is PyTorch and MONAI on V100 and A100 GPUs; generating a 512³ volume needs about 58 GB of GPU memory. On out-of-distribution data the shared VAE matched dataset-specific VAEs on LPIPS, SSIM and PSNR at zero additional training cost, against 619–672 GPU hours for the dedicated models. Evaluated on the unseen autoPET 2023 whole-body cohort against baselines retrained on the same corpus, its diffusion model reached an average FID of 6.083, versus 12.379 for LDM, 13.757 for HA-GAN and 22.608 for DDPM.

#Applications

The primary use is data augmentation where annotated volumes are scarce. Training segmentation models on real plus MAISI-generated data raised mean Dice across five tumor types — liver, lung, pancreas, colon and bone lesion — by 4% on average for mask-conditioned generation and 6.5% for inpainting on liver, lung and pancreas. Because synthetic volumes carry no patient identity, they also offer a route to circulating training material where the underlying scans cannot leave the institution. The conditioning interface also supports controlled experiments: organ size, lesion presence and field of view can be varied to probe how a detector handles presentations rare in real cohorts. Conditioning masks usually come from a whole-body segmentation model such as VISTA3D rather than being drawn by hand.

#Impact

MAISI showed that whole-volume, variable-geometry 3D CT synthesis is tractable without per-dataset generators. It ships as a MONAI Model Zoo bundle (maisi_ct_generative) that runs from random noise with one command, as an NGC model resource, and formerly as a hosted NIM endpoint, since deprecated; NVIDIA's later rectified-flow successor redistributes the same v1 checkpoints as ddpm-ct inside the NV-Generate-CT repository. The bundle's code is Apache-2.0 while its weights restrict third-party use to research and evaluation, and the model card states the model is for research rather than clinical use. The training corpus is given as volume counts rather than a released manifest, and the authors note that whether generated anatomy reflects demographic variation across age, ethnicity and sex is not extensively validated — the very gap synthetic data is often meant to fill.