CXR Foundation

CXR Foundation is a chest X-ray embedding model developed by Google Research and released as part of the Health AI Developer Foundations (HAI-DEF) collection. It converts chest radiographs into embeddings that accelerate the development of AI tools for chest X-ray analysis, letting developers build classifiers with less labeled data and compute than training from scratch. Its current version is built on ELIXR, an approach that aligns a radiology vision encoder with a language model to enable both image-only and image-text applications.

The model tackles two recurring problems in radiology AI: labeled chest X-rays are costly to curate, and many useful tasks benefit from connecting images to natural-language concepts. By producing rich image embeddings and, through its contrastive text pathway, projecting images and text into a shared space, CXR Foundation supports data-efficient supervised classification, zero-shot classification from text prompts, and semantic image retrieval within a single framework.

CXR Foundation's ELIXR-based version 2.0.0 was created on August 2, 2024. The underlying methods are described in "ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoders" (Xu et al., 2023), with earlier data-efficient transfer learning work reported in Radiology. Google has designated CXR Foundation a legacy model that remains available for existing applications.

#Key Features

• Two embedding types: ELIXR v2.0 provides 32x768 dimensional image embeddings for detailed visual features, while an ELIXR-contrastive text pathway provides 32x128 dimensional embeddings that place images and text in a shared semantic space.
• Zero-shot classification: The shared image-text space allows findings to be classified from natural-language prompts without task-specific training labels.
• Semantic image retrieval: Image-text alignment enables searching radiograph archives by textual description or by example image.
• Data-efficient supervised learning: Downstream classifiers trained on the embeddings reach strong accuracy with modest labeled datasets.
• Multi-source training: Built from radiographs spanning US and Indian datasets, with labels derived from reports and validated with model-assisted and radiologist adjudication.

#Technical Details

CXR Foundation pairs an EfficientNet-L2 vision encoder with a BERT text encoder, trained using a combination of supervised contrastive, CLIP, and BLIP-2 objectives to align image and language representations following the ELIXR design. Training used 821,544 chest X-rays: 243,324 from MIMIC-CXR, 165,182 from a US private dataset, and 485,082 from a set of Indian hospitals, with labels derived from radiology reports and validated using a large language model with thoracic-radiologist adjudication. On CheXpert benchmarks, the model achieved a mean AUC of 0.898 for data-efficient classification across five conditions, 0.846 for zero-shot classification across 13 findings, and 0.76 NDCG@5 for semantic image retrieval.

#Applications

CXR Foundation is aimed at researchers and developers building chest radiograph analysis tools. Its image embeddings support data-efficient training of classifiers for findings such as cardiomegaly, effusion, or atelectasis, while its text-aligned embeddings enable zero-shot detection from prompts and semantic retrieval across large X-ray collections. This makes it useful for prototyping triage tools, curating and searching imaging datasets, and conducting retrospective research where assembling large annotated sets would otherwise be prohibitive.

#Impact

CXR Foundation extended Google's HAI-DEF strategy of releasing reusable embedding models to radiology, and its ELIXR foundation showed that aligning vision encoders with language models unlocks zero-shot and retrieval capabilities beyond fixed-label classification. Alongside Path Foundation and Derm Foundation, it reinforced the value of domain-specific pretraining for reducing downstream data and compute needs. Its limitations warrant attention: it is released for research and development and has not been cleared for clinical use; reported metrics come from specific benchmarks and datasets, so performance may vary across institutions, scanners, and populations; and Google now labels it a legacy model maintained for existing applications.