Derm Foundation

Derm Foundation is a dermatology image embedding model developed by Google Research and released as part of the Health AI Developer Foundations (HAI-DEF) collection. It is designed as a reusable feature extractor: given a photo of skin, it produces a dense embedding that captures features relevant to dermatological analysis, which developers can then use to train downstream classifiers with substantially less data and compute than building a model from scratch.

The model targets a practical challenge in dermatology AI. Skin presentations vary enormously across conditions, skin tones, imaging devices, and lighting, and labeled clinical images are difficult to assemble at scale. By pretraining a general-purpose skin-image encoder, Derm Foundation lets teams build tools for tasks such as classifying clinical conditions and scoring disease severity on top of a fixed embedding, rather than training large networks end to end.

Derm Foundation was created on December 19, 2023, and is documented alongside its sibling models in the "Health AI Developer Foundations" paper (Kiraly et al., 2024). It is distributed with a model card and inference code on Hugging Face and GitHub under the HAI-DEF terms of use. Google has since designated it a legacy model, directing new projects toward its successor MedSigLIP while keeping Derm Foundation available for existing applications.

#Key Features

• High-dimensional skin embeddings: Maps 448x448 pixel dermatology images to 6144-dimensional float vectors that encode dense, condition-relevant visual features.
• Two-stage training: Combines contrastive learning on large-scale image-text pairs from the public web with supervised fine-tuning for skin-condition classification, blending broad visual pretraining with clinical specialization.
• Data-efficient downstream modeling: Linear classifiers trained on Derm Foundation embeddings showed roughly 10-15% higher accuracy than standard BiT-M baselines across a range of training-data proportions.
• Device and source diversity: Fine-tuning drew on tele-dermatology data from the US and Colombia, an Australian skin cancer dataset, and public images captured with smartphones, cameras, and dermatoscopes.
• Developer-oriented release: Ships with a documented model card, Hugging Face weights, and example code to accelerate applied dermatology ML.

#Technical Details

Derm Foundation uses a BiT-M ResNet101x3 convolutional backbone. Training proceeded in two stages: first, contrastive learning aligned images with associated text drawn from a large collection of public web image-text pairs, following an approach in the lineage of ConVIRT and Big Transfer (BiT); second, the image encoder was fine-tuned in a supervised fashion for condition classification using clinical dermatology datasets. The model accepts 448x448 pixel PNG images and outputs a 6144-dimensional embedding. In benchmarking, linear probes on these embeddings delivered consistent accuracy gains over standard BiT-M features across varying amounts of downstream training data, illustrating the transfer value of domain-specific pretraining.

#Applications

Derm Foundation is aimed at researchers and developers building dermatology AI tools. Downstream tasks include classifying clinical conditions such as psoriasis, melanoma, and dermatitis, and scoring the severity or progression of skin conditions. Because it exposes a frozen embedding, teams can prototype classifiers rapidly, run retrospective studies economically, and support triage or teledermatology research workflows without assembling the very large annotated datasets a full training run would require. It is especially useful for groups with limited labeled data or compute.

#Impact

As one of the first models in Google's HAI-DEF collection, Derm Foundation helped establish the pattern of releasing domain-specific embedding models that developers assemble into applications, rather than shipping fixed diagnostic systems. Together with Path Foundation and CXR Foundation, it showed that pretrained health-image embeddings can cut the data and compute needed for downstream tasks. Its limitations are important: it is released for research and development and has not been cleared for clinical use; embeddings reflect the conditions, devices, and populations in its training data, so performance can vary across skin tones and imaging settings; and Google now positions the multimodal MedSigLIP model as its recommended successor for new work.