PRETI

PRETI is a self-supervised foundation model for colour fundus photography that conditions its representations on routinely collected patient metadata. Recent retinal foundation models have improved by pairing images with clinical reports, but reports are expensive to obtain, inconsistently written, and often unavailable outside academic centres. Age and sex, by contrast, accompany essentially every retinal examination and carry real signal — the retina's vasculature and optic disc change measurably with age, and disease prevalence differs by sex — yet prior models discard them during pretraining.

The model addresses this with three ideas. Learnable Metadata Embedding prepends trainable age and sex embeddings to the image patch tokens and supervises them with an auxiliary objective, so the metadata representation co-adapts with visual features instead of being a fixed side channel. Patient-level pairing treats the two eyes of one individual (and repeat visits) as a positive pair, teaching the encoder to ignore camera, illumination, and laterality differences that are not clinically meaningful. Retina-Aware Adaptive Masking confines masked-autoencoder masking to the circular retinal region rather than the black surround, and anneals the masking ratio downward over training so the model first learns global structure and later fine pathological detail.

PRETI was developed at Yonsei University with the medical AI company Mediwhale and received an Early Accept at MICCAI 2025. It sits in a growing family of retinal foundation models alongside RETFound, EyeFound, and EyeCLIP.

#Key Features

• Learnable Metadata Embedding: Age and sex enter as trainable tokens trained with an auxiliary loss combining RMSE for age and cross-entropy for sex, letting patient context shape the visual representation during pretraining rather than only at fine-tuning.
• Patient-level image pairing: Left–right eye pairs (including self-pairs) from the same individual anchor a consistency objective that suppresses non-clinical variation across acquisitions.
• Retina-Aware Adaptive Masking: Masking is applied only inside the retinal field of view, with the ratio decayed on a cosine schedule from 0.985 to 0.85 across training.
• Systemic biomarker prediction: Beyond ocular disease, the encoder predicts coronary artery calcium score and estimated glomerular filtration rate from the fundus alone, extending the model to cardiovascular and renal risk.
• Strong small-encoder performance: A ViT-S PRETI encoder outperforms a ViT-L RETFound on several in-house tasks, an efficiency advantage for deployment on modest hardware.

#Technical Details

The encoder is a ViT-S or ViT-B initialised from ImageNet-1K weights, paired with a ViT-S decoder under a masked autoencoder objective augmented with perceptual reconstruction, metadata, and consistency losses. Pretraining used 1,017,549 colour fundus photographs from 292,006 patients across six medical institutions, including the UK Biobank, at 224×224 resolution for 300 epochs on eight H100 GPUs. On an in-house evaluation set, PRETI (ViT-B) reaches AUROC 0.982 for diabetic retinopathy, 0.777 for glaucoma, and 0.772 for age-related macular degeneration, against 0.969, 0.747, and 0.695 for RETFound (ViT-L); on biomarkers it reaches 0.899 for coronary artery calcium and 0.699 for eGFR, the latter well ahead of RETFound's 0.582. Public benchmarks span APTOS2019, MESSIDOR-2, GF, PAPILA, Retina, JSIEC, and IDRID, where PRETI (ViT-B) is competitive with report-supervised models such as RET-CLIP and UrFound without using any text. Ablations show that removing either the metadata embedding or the adaptive masking degrades both AUROC and AUPRC.

#Applications

The model is intended as a general-purpose retinal encoder for screening and triage: grading diabetic retinopathy, detecting glaucoma and macular degeneration, and estimating systemic cardiovascular and kidney risk from a single inexpensive photograph. Because metadata conditioning uses only age and sex, the approach fits ophthalmic and optometric settings where structured reports do not exist — community screening programmes, primary care, and teleophthalmology — rather than requiring a paired clinical-text corpus.

#Impact

PRETI makes the case that cheap, universally available patient attributes are a practical substitute for expensive report supervision in medical imaging pretraining, a result that generalises beyond ophthalmology. Training code is public and a ViT-B checkpoint is distributed via a download link in the repository README. Two constraints bound the findings: the pretraining corpus and the principal disease and biomarker evaluations are in-house and not redistributable, so those numbers cannot be independently reproduced, and the repository carries no license file, leaving reuse terms unstated.