CLEAR

A staging CT can run to seven hundred axial slices, and for most clinical questions only a handful of them carry the answer. The prevailing radiology recipe absorbs that cost by training an end-to-end 3D network for each new task, which means convolving over every voxel of healthy anatomy and, in practice, paying a radiologist to select scans and delineate lesions first. CLEAR (Contrastive Learning-based Embeddings for Attention-based Radiology) takes the route computational pathology settled on instead: embed each 2D axial slice once with a frozen pretrained encoder, then train only an attention-based multiple-instance pooler to decide which slices matter for the endpoint at hand. Scan selection and lesion delineation become something the attention weights do rather than something a human does.

The framework's pretraining half is lesion-enhanced contrastive learning (LeCL). Plain contrastive pretraining on axial slices learns mostly anatomy, because anatomy is what varies most between random crops. LeCL biases the crops instead: on slices that carry a lesion bounding box, one branch of the contrastive pair sees a lesion-centred crop while the other sees the full slice, so the invariance the encoder learns is anchored to the abnormality rather than to the surrounding organ. A weighting term λ additionally pushes annotated slices apart from the rest of the batch in the loss denominator. The boxes shape the sampling, never the targets, so no task labels enter pretraining.

CLEAR was developed at the Else Kröner Fresenius Center for Digital Health of TU Dresden's Carl Gustav Carus medical faculty, with RWTH Aachen University, and posted as a preprint in November 2024 before appearing at MICCAI 2025.

#Key Features

• Frozen encoder, trained pooler: downstream use follows the conventional self-supervised linear protocol — backbone weights are frozen and only the ABMIL adapter and classification head are fit, so one feature extraction serves any number of endpoints.
• Abnormality-driven pretraining: LeCL turns DeepLesion's lesion boxes into a sampling constraint on the contrastive pairs, giving representations organised by pathology instead of by anatomical location.
• Architecture comparison across three families: vision transformers, vision state space models (VMamba) and gated convolutional networks (MambaOut) were pretrained identically; the gated CNN won on all three endpoints, beating ViT by 2.4, 2.6 and 6.8 AUC points.
• Interpretable slice attention: the per-slice attention distribution concentrates on lesion-bearing slices and favours the intensity window in which the lesion is better depicted, which makes the evidence behind a prediction inspectable.
• Modest compute footprint: pretraining each variant took under two days on four A100 GPUs, and every encoder stays under 100 million parameters.

#Technical Details

Pretraining used a MoCo-v3 base with the LeCL objective over 873,849 axial CT slices from DeepLesion (10,224 unlabelled and 3,538 labelled scans), for 100 epochs at a learning rate of 1e-4, batch size 2048 for VMamba and MambaOut and 1024 for the convolutional-stem ViT-B. Slices were clipped to abdominal (-175 to 275 HU) and lung (-1500 to 500 HU) windows. Downstream, up to 700 512×512 slices per patient are embedded and pooled by a gated attention layer followed by an MLP head.

Against BiomedCLIP, CT-CLIP, Merlin and SAM2 on frozen features, the MambaOut LeCL-0 configuration reached a mean AUC of 78.4 across eight anatomical sites on held-out DeepLesion patients, against 77.6 for BiomedCLIP. On external cohorts it held up: 66.9 mean AUC over eight abnormality labels in 3,630 RadChest chest CT patients (+1.9 over BiomedCLIP, +5.8 on emphysema alone), and AUC 68.3 / AUPRC 78.3 / F1 72.3 for low-versus-high staging in 422 NSCLC-Radiomics patients. The λ ablation is instructive: λ = 0 was consistently best and raising it to 5 cost up to 4 AUC points, so the gain comes from where the crops land rather than from reweighting the loss.

#Applications

The framework suits retrospective radiology cohorts where labels exist at the patient level — stage, treatment response, outcome — but per-lesion annotation does not. Because features are extracted once and frozen, a group can embed an archive of CT studies and then fit cheap attention heads per question — the workflow that made whole-slide pathology foundation models practical. The attention maps double as a triage signal, pointing to the slices a reviewer should look at.

#Impact

CLEAR is a demonstration that the pathology playbook — task-agnostic frozen features plus weakly supervised aggregation — transfers to CT without volumetric pretraining, using far less data and compute than the 3D vision-language models it is measured against. The released artifacts are narrower than the study: the repository ships only the MambaOut LeCL-0 and LeCL-1 checkpoints under GPL-3.0 via Git LFS, leaving the ViT and VMamba variants and the other λ settings undistributed, and there is no hosted API or model hub mirror. The authors are candid that absolute performance on the external cohorts remains modest and that radiology representation learning has room to run.