CoralBay

A CT scan is not a picture. Its voxels carry Hounsfield Units — physical tissue densities — and a radiologist reads the same volume several times over, once through a lung window, once through soft tissue, once through bone: narrow the window and high-density structures clip away, widen it and soft tissue flattens into grey. The volume is also genuinely three-dimensional. Vision models pretrained on natural images encode neither property, and the usual workaround of treating a scan as a stack of independent 2D slices discards the spatial continuity that makes a few-voxel lesion visible at all.

CoralBay, released by kaiko.ai in June 2026, addresses both directly. It extends DINO's teacher–student self-distillation from 2D images to native 3D volumes using a hierarchical 3D Swin Transformer encoder built along the design principles of Swin UNETR, and it treats windowing as an augmentation rather than a preprocessing decision: each training crop is rendered through a randomly sampled clinically relevant HU range — full CT range, whole body, lung, abdomen, liver, brain — so the encoder is pushed toward features that hold across all of them.

The second design choice is where the distillation loss is applied. Instead of distilling only the final-stage embedding, CoralBay applies 3D adaptive average pooling at every stage of the Swin hierarchy and concatenates the per-resolution vectors before the DINO projection head. One scale-aware target then supervises coarse anatomy and fine structure together, letting the framework skip the compute-intensive iBOT patch-level loss of DINOv2 while still learning the local detail voxel-wise segmentation depends on.

#Key Features

• Native 3D self-distillation: Teacher and student process two 96x96x96 global crops and six 48x48x48 local crops from a 115-voxel subvolume, extending DINO's local-to-global principle into three dimensions.
• Random Hounsfield-Unit windowing: Eight clinically relevant HU ranges are sampled at random during pretraining, so one backbone stays usable across the windowing conventions of different protocols.
• Multi-scale feature concatenation: Pooled features from every Swin stage form one aggregated vector, letting the DINO loss supervise organ-level context and vessel- or tumour-margin detail together.
• Sliding-window scan inference: Whole scans are split into 96-voxel cubes, encoded independently and stitched back in spatial alignment — pooled for classification, or decoded into voxel-wise masks.
• Data-efficient pretraining: The recipe runs on roughly 11,000 unlabeled volumes from public sources, far fewer than comparable 3D corpora.

#Technical Details

Two checkpoints were trained: CoralBayU96B at 53.2M parameters and CoralBayU96H at 847M, of which only CoralBayU96H was released. Both ran for 2,000 epochs with AdamW on CORID (Combination Of Radiology Image Data), a collection balanced across chest, abdomen, lung and head-and-neck and drawn from AbdomenAtlas Mini 1.0 (5,195 volumes), STOIC 2021 (~2,000), Stony Brook COVID-19 (1,384), LIDC-IDRI (1,018), LUNA16 (888), HNSCC (627) and TCGA-HNSC (227).

Evaluation runs through eva, kaiko.ai's open-source oncology evaluation framework, across 11 datasets. Under linear probing, CoralBayU96H reaches 0.99 accuracy on OrganMNIST3D, 0.90 on CC-CCII COVID-19 classification, 0.80 on NoduleMNIST3D and 0.74 AUROC on LUNA25, plus 0.93 AUROC on the LUNA25 open development leaderboard. For segmentation the encoder stays frozen and only a Swin UNETR decoder is trained, reduced from 313M to 22.8M parameters by 1x1x1 convolutions: Dice reaches 0.97 on CHAOS, 0.91 on FLARE22, 0.85 on WORD, 0.82 on BTCV and 0.81 on both LiTS17 liver tumour and KiTS23 — level with VoCo, which pretrained on 160,000 volumes, from under 7% of the data. Full fine-tuning raises classification to 0.91/1.00/0.91/0.82 and brings segmentation within two points of nnU-NetV2 — ahead on KiTS23, level on LiTS17, two behind on BTCV and MSD Pancreas; MSD Pancreas tumour remains the weakest task, at 0.67 frozen and 0.73 fine-tuned.

#Applications

The released checkpoint is a general-purpose CT encoder rather than a task model. Frozen, it supplies scan-level embeddings for triage and diagnostic classification — organ identification, nodule malignancy scoring, disease severity — and dense features for multi-organ and lesion segmentation behind a lightweight decoder. A group with a few hundred annotated volumes can therefore fit a probe or a small decoder instead of a full segmentation network, which is where the label-efficiency ablations show the largest gains: on tumour tasks, self-supervised initialisation beats a heavily tuned nnU-Net in low-data settings, though on simpler organ segmentation nnU-Net stays competitive.

#Impact

CoralBay extends kaiko.ai's open-model line from pathology into radiology, following the Midnight tile encoders and the eva framework in the same pattern of open weights plus public evaluation. Its central claim is about data rather than scale: a 3D pretraining recipe on 11,000 volumes matches contrastive frameworks trained on an order of magnitude more, lowering the barrier for groups without hospital-scale imaging archives. kaiko.ai has also contributed a 3D radiology leaderboard to eva, standardising the protocol such comparisons otherwise lack — frozen backbone, fixed decoder, averaged over five runs. The work is a preprint awaiting peer review, the MIT-licensed weights are intended for research use only and carry no regulatory clearance, and coverage is CT alone.