Loading…
Dense self-supervised pretraining that learns pixel- and voxel-level features for segmenting cardiac MRI and abdominal CT from very few labels.
No providers recorded yet. Browse all providers
Most self-supervised pretraining for medical images learns one descriptor per scan, the wrong granularity for segmentation, which needs a label at every pixel. Dense self-supervised methods address this by contrasting pixel features drawn from two partially overlapping crops of the same scan — but that construction is self-limiting. The crops have to keep overlapping, so spatial augmentation must stay mild, and the similarity matrix grows with the square of the number of pixels, so only a sparse sample of locations can be used at all.
Pix2Rep-v2 removes the overlap constraint by making the supervision geometric rather than comparative. Each patch becomes two views under independent photometric augmentations, and one random spatial transformation — flips, rotations, zooms, B-spline elastic deformations — is applied asymmetrically: one view is warped then encoded, the other encoded then warped. If the representation is genuinely equivariant, the two dense feature maps must agree pixel for pixel. The training loss is the Barlow Twins redundancy-reduction objective on the cross-correlation matrix of the projected features, whose size depends on the embedding dimension rather than the pixel count, so every pixel in the batch can contribute without a negative-sample blowup.
The framework comes from the LTCI laboratory at Télécom Paris, Institut Polytechnique de Paris, with clinical cardiology co-authors at Hôpital Universitaire Lariboisière (AP-HP) and Université Paris Cité. Posted in September 2026 and accepted at MICCAI 2026, it succeeds the same group's earlier Pix2Rep, replacing that model's contrastive loss and adding 3D, voxel-level and training-free capability.
The encoder–decoder backbone is deliberately interchangeable: 2D experiments use U-Net and Swin UNETR with an embedding dimension of 1024 and a 256-dimensional projector, while 3D experiments use the implicit U-Net at 256 and 128 dimensions. Cardiac pretraining pools the 509 training subjects of ACDC, M&Ms and M&Ms-2 cine MRI without labels; the abdominal CT configuration pretrains on 400 AMOS subjects plus 1,900 unlabeled CT scans. The 3D objective is evaluated at 2^17 sampled coordinates per patch, over a hundred times the roughly 1,024 points typical of contrastive dense methods. Pretraining takes under a day on four H100 GPUs for 200 epochs; fine-tuning runs on a single V100. With 1% of M&Ms-2 labels, the fine-tuned U-Net variant gains 9.3 Dice points over Pix2Rep and 15.0 over BT-UNet, and the in-context variant at the same support size beats every fine-tuned baseline while sitting roughly 35 Dice points above in-context ProtoSAM. On AMOS multi-organ CT it matches a from-scratch implicit U-Net using five times fewer annotated scans, and in zero-shot cine propagation it outperforms SAM 3 and approaches MedSAM2.
The target user is an imaging group holding plenty of unlabeled scans and very few delineations — a new scanner, a new cohort, a structure nobody has annotated yet. Because pretraining is cheap and local, a site can run it on its own raw archive and then choose how much annotation to spend: full fine-tuning, linear probing on a frozen encoder, or the in-context route that needs a support set and no training. The demonstrated tasks are cardiac chamber and myocardium segmentation, multi-organ abdominal CT segmentation, and propagating end-diastolic annotations to end-systole in cine series.
Pix2Rep-v2 argues for a second path alongside generalist medical vision models: instead of one large checkpoint pretrained on heterogeneous data, a compact recipe a single lab can run on premise that still reaches few-shot behaviour competitive with fine-tuned foundation models. Its evaluation is correspondingly narrow — the authors claim no generality beyond dense cardiac MRI and abdominal CT tasks, and the pretraining corpora are hundreds of scans rather than web-scale. The work is a preprint of a conference paper awaiting proceedings publication. The implementation is released under Apache 2.0, but the pretrained U-Net and Swin UNETR checkpoints are announced as forthcoming rather than posted, and the dataset split files the repository refers to are unpublished, so reproducing the reported numbers currently requires pretraining from scratch.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.