MINIM

MINIM (Medical Image-text geNeratIve Model) is a generative foundation model that synthesizes realistic medical images of multiple organs across several imaging modalities directly from free-text instructions. Rather than predicting diagnosis from an existing scan, it tackles the inverse problem: producing synthetic images on demand to expand scarce, privacy-constrained, or imbalanced datasets. Clinical imaging AI is bounded less by model capacity than by data, and the shortage is sharpest where it matters most — rare conditions, underrepresented populations, and newer modalities.

MINIM was developed by Jinzhuo Wang and colleagues at Peking University, with the Institute for AI in Medicine at Macau University of Science and Technology, Wenzhou Medical University and collaborating institutions, and published in Nature Medicine in December 2024. Where earlier text-conditioned medical generators such as RoentGen were fitted to a single modality, one MINIM checkpoint spans optical coherence tomography (OCT), fundus photography, chest X-ray, chest CT and brain MRI, and the authors position it as a step toward generalist medical AI.

A defining feature is the self-improving training loop. After supervised diffusion pretraining, the model is refined with reinforcement learning from radiologist feedback, raising the realism and clinical plausibility of its outputs; following that fine-tuning, 91% of MINIM-generated OCT images received the highest quality rating from clinicians. The same mechanism lets the model absorb an entirely new domain: incorporating a private breast MRI cohort improved generative quality on that modality rather than degrading the rest.

#Key Features

• Text-to-image generation across modalities: A single model synthesizes OCT, fundus, chest X-ray, chest CT and brain MRI images conditioned on natural-language prompts describing organ and pathology.
• Self-improving via reinforcement learning: Radiologist ratings of image quality and prompt agreement act as the reward signal, measurably increasing the rated quality of synthetic images over the base diffusion model.
• Extends to unseen domains: Adding a previously unseen breast MRI dataset raised generative performance on that domain, evidence the model improves as new data arrives rather than needing a fresh model per task.
• Dataset augmentation that boosts downstream tasks: Mixing synthetic with real images improves diagnostics, report generation and self-supervised pretraining, with average gains of 12% (ophthalmic), 15% (chest), 13% (brain) and 17% (breast).
• Clinically meaningful applications: Synthetic augmentation supports prediction of HER2-positive breast cancer from MRI and identification of targeted-therapy-sensitive EGFR mutations from lung cancer CT.

#Technical Details

MINIM is a latent text-to-image diffusion model built on a Stable Diffusion-style framework. A BERT tokenizer encodes the modality label and the free-text description separately, and the paired embeddings condition a U-Net denoiser through cross-attention as the model learns to reverse a linearly scheduled Gaussian noising process. The released configuration initializes from pretrained diffusion weights and trains at 512×512 resolution for 20,000 steps, where ablations report an FID of 57.91, an Inception Score of 5.89 and MS-SSIM of 0.18; for ophthalmic OCT the reported figures are FID 65.3, IS 5.7 ± 0.42 and MS-SSIM 0.16 ± 0.03. Comparisons were run against Imagen, DALL·E, GigaGAN and StyleGAN-T fine-tuned on the same data, with blinded clinician review alongside the objective metrics and image-image / image-text retrieval scores.

Downstream evaluation used off-the-shelf architectures rather than bespoke ones: a Swin Transformer classifier augmented with 100–500 synthetic images per diagnostic category, a CLIP+GPT-2 report generator scored on BLEU, CIDEr and ROUGE-L, and a DenseNet-121 trained self-supervised on synthetic data. Chest CT diagnostic accuracy rose from 0.58 with real data alone to 0.79 with a mixed corpus, and mutation-classification gains scaled with the synthetic-to-real ratio: EGFR prediction from lung CT climbed from 81.5% to 91.2% at 1:1 and 95.4% at 5:1, while HER2 status prediction from breast MRI rose from 79.2% to 94.0%.

#Applications

MINIM is intended for medical-AI researchers and clinical informaticians who need to enlarge or rebalance training datasets without collecting and de-identifying additional patient scans. Synthetic images can pad rare diagnostic categories, seed self-supervised pretraining, and supply paired image-report data for automated radiology report generation. The HER2 and EGFR results point at a more consequential use: both markers normally require biopsy and molecular assay, and a large retrospective simulation suggested that accurate non-invasive triage from routine imaging could translate into improved five-year survival.

#Impact

Published in Nature Medicine and cited well over a hundred times within its first year, MINIM is among the more prominent peer-reviewed demonstrations that synthetic medical images can measurably improve clinical models rather than merely look convincing. Its limitations are concrete. All results are retrospective and in silico, with no prospective validation; the authors note overfitting risk and degraded image-text alignment on longer prompts; and generated images can inherit artifacts or biases from the training distribution. The clinical image-text training corpus is private and unreleased, the code repository carries no license file, and weights are distributed as a prebuilt executable through a third-party file host rather than a versioned model hub, with no model card or datasheet — a real barrier to independent reproduction of a model whose central argument is that synthetic data can be trusted in the clinic.