DermINO

Skin disease affects a very large share of the population while dermatologists remain scarce outside major cities, which makes dermatology one of the most obvious targets for image-based clinical AI. The models built for it so far have mostly been narrow: trained on a single labeled dataset for a single task, such as melanoma-versus-benign classification, and brittle when moved to a new camera, a new clinic, or a different distribution of skin tones.

DermINO is a dermatology vision foundation model that attacks this with a hybrid pretraining framework. It was developed by a collaboration led from the China-Japan Friendship Hospital in Beijing with Xidian University, Microsoft Research Asia, and clinical and academic partners, and released as a preprint in August 2025. Rather than choosing between purely self-supervised pretraining, which ignores the diagnostic labels that do exist, and supervised training, which is capped by how much labeled dermatology data anyone has, DermINO combines both: DINO-style image-level and patch-level self-supervision run alongside a semi-supervised objective over partially annotated data, with disease prototypes initialized from a medical language model's embeddings so the label space carries clinical structure from the start.

The result is a single backbone evaluated across 20 external datasets spanning six task families, positioning it alongside — and benchmarked against — other dermatology encoders such as Google's Derm Foundation.

#Key Features

• Hybrid self- and semi-supervised pretraining: Image-level DINO distillation and patch-level masked image modeling are trained jointly with a supervised term over the labeled subset, so annotated and unannotated images contribute to the same encoder.
• Knowledge-guided prototype initialization: Class prototypes are initialized from CODER medical-language-model embeddings of disease names, injecting relationships between conditions into the representation instead of treating diagnoses as unstructured one-hot targets.
• Six task families from one encoder: Malignancy assessment, severity grading, multi-category diagnosis, image captioning, and lesion segmentation are all evaluated, covering both high-level clinical judgment and pixel-level tasks.
• Demographic robustness testing: Performance is reported across Fitzpatrick skin types I-VI and both sexes, with the authors reporting stable accuracy across those strata.
• Federated-learning compatibility: The model was evaluated in privacy-preserving federated settings across three datasets, relevant where dermatology images cannot leave the institution that collected them.

#Technical Details

The backbone is a ViT-Base encoder with roughly 86 million parameters per network in the teacher-student arrangement. Pretraining used 432,776 curated images from three streams: 232,215 proprietary images (53.7%) from the China-Japan Friendship Hospital, 104,562 public images (24.2%) from repositories including ISIC, SCIN, SD-198, PAD-UFES-20 and DermNet, and 95,999 web-sourced images (22.2%). Evaluation spans 20 datasets: five for malignancy assessment, three for severity grading, six for multi-category diagnosis, one for captioning, and five for segmentation. Against the strongest baselines, primarily PanDerm and MedImageInsight, the reported gains are 2.31% retrieval AUROC and 4.03% F1 on malignancy assessment, 4.46% AUROC and 11.01% F1 on severity grading, 1.38% AUROC and 6.26% F1 on multi-category diagnosis, and 2.54% Jaccard with 1.0% Dice on segmentation. A blinded reader study with 23 dermatology image specialists on 119 images reported 95.79% diagnostic accuracy for the model against a 73.66% clinician average, with AI assistance raising clinician performance by 17.21%.

#Applications

The intended uses are triage and decision support in settings where dermatological expertise is thin: flagging likely malignancy from clinical or dermoscopic photographs, grading severity for conditions such as acne where longitudinal consistency matters, producing draft descriptive captions, and delineating lesion boundaries for measurement. For research groups, the value is a pretrained dermatology encoder that transfers to new labeled cohorts with modest data, and the federated-learning results make it applicable to multi-hospital collaborations constrained by patient-privacy rules.

#Impact

DermINO's central methodological claim — that partially labeled medical corpora are best exploited by fusing self-supervised and semi-supervised objectives rather than pretraining then fine-tuning in sequence — is a pattern with broad applicability wherever a domain has abundant images and patchy annotation. The reader study is unusually large for a dermatology preprint and its fairness analysis across Fitzpatrick types addresses a well-documented weakness in skin-lesion models. Several limits are worth weighing: the work is a preprint and has not completed peer review; the released code and ViT-B/14 checkpoint carry no licence, leaving the terms of reuse undefined; over half the pretraining corpus is proprietary single-institution data; the authors note thin coverage of rare genetic and systemic disease and incomplete fairness evaluation across age groups; and they flag the risk of clinician over-reliance observed in the assisted-reading arm.