DoDNet

The LiTS challenge labels livers and liver tumors. KiTS labels kidneys and kidney tumors. Pool the two and the same anatomy is foreground in one scan and unannotated background in the next, so a network trained on the union learns to suppress exactly what the other dataset asks it to find. This partial-labeling problem is why abdominal CT segmentation long defaulted to one network per organ.

DoDNet answers it by making the segmentation head itself conditional. A shared 3D encoder-decoder produces a task-agnostic feature map, and a small controller generates the kernels of a lightweight head on the fly, conditioned on both the input volume and a one-hot vector naming the task to run. Because the head is generated rather than fixed after training, one set of weights covers liver, kidney, hepatic vessel, pancreas, colon tumor, lung tumor and spleen; asking for a different target changes the task vector, not the network. Losses are computed only over annotated targets, so an unlabeled organ is simply ignored instead of being taught as background.

Jianpeng Zhang, Yutong Xie and Yong Xia at Northwestern Polytechnical University, with Chunhua Shen at the University of Adelaide, presented the work at CVPR 2021. To train it they assembled MOTS, a partially labeled abdominal CT corpus built from LiTS, KiTS and the Medical Segmentation Decathlon, then transferred the pretrained weights to a downstream multi-organ task — the framing later renal-pathology work such as Omni-Seg inherited, along with this codebase's checkpoint naming.

#Key Features

• Task-conditioned dynamic head: A single convolutional controller takes globally pooled image features concatenated with the task vector and emits the 162 kernel parameters of a three-layer head, so the filters adapt to both the scan and the requested target.
• One backbone, seven targets: Liver, kidney, hepatic vessel, pancreas, colon tumor, lung tumor and spleen are served by one encoder-decoder, replacing seven task-specific networks at roughly one-seventh the parameters.
• Learning from partial labels: Predictions for targets a scan does not annotate are excluded from the loss, avoiding the standard trick of merging unlabeled organs into the background.
• MOTS training corpus: 1,155 3D abdominal CT scans, 920 for training and 235 for testing, assembled from seven single-task challenge datasets and resampled to a common 1.5 x 0.8 x 0.8 mm voxel size.
• Transferable pretrained weights: The MOTS-pretrained encoder-decoder is released as a checkpoint, intended as an initialization for volumetric segmentation tasks with limited annotation.

#Technical Details

The backbone is a U-shaped 3D network of residual blocks with 3x3x3 convolutions, group normalization and weight standardization, four downsampling stages, and 32 filters in the first layer doubling at each stage; the decoder returns an eight-channel pre-segmentation feature map. The generated head is three stacked 1x1x1 convolutions — eight channels, eight channels, then two, one for the organ and one for its tumors. Training used SGD with momentum 0.99, a polynomial schedule from a 0.01 learning rate over 1,000 epochs, 64x192x192 sub-volumes, and a combined Dice and binary cross-entropy objective; inference is a sliding window. The whole model is 17.3M parameters, against 121.0M for seven separate networks and 18.9M for a shared-encoder multi-head baseline, and the dynamic head costs almost nothing per task. On the MOTS test set it reaches 75.64% average Dice and 19.50 average Hausdorff distance over 11 organ and tumor categories. Ablation shows the task condition carries the method: removing it drops average Dice from 71.67% to 51.80% on the validation split, while removing the image-feature input costs about half a point. Fine-tuning from MOTS weights on the BCV multi-organ benchmark raised average Dice from 85.30% to 86.44% and cut mean surface distance from 1.46 to 1.17 mm relative to training from scratch.

#Applications

The practical draw is deployment economics for abdominal CT reading: a group that would otherwise maintain seven organ-specific models keeps one checkpoint and selects a target at call time, with contouring, volumetry and surgical-planning pipelines drawing on the same weights. The transfer result matters more for groups with small annotated cohorts, where initializing a 3D segmentation network from MOTS converges faster and ends higher than initializing from scratch or from any single challenge dataset.

#Impact

DoDNet made task-conditioned dynamic filters a standard way to train one segmentation network across datasets that disagree about what is labeled, and the design propagated directly into downstream medical segmentation work, including renal pathology models that still ship weights under this codebase's naming. Its limits are equally clear: the task encoding is a closed set of seven, so an eighth organ requires retraining rather than a new task vector, and the BCV result is fine-tuned transfer, not zero-shot generalization. Reproduction is harder than it should be — the preprocessed MOTS corpus was never distributed, leaving users to rebuild it from seven upstream challenges with separate access terms. The code is GPL-3.0; no separate license is stated for the released weights.