RoMedFormer

Radiation therapy for anal, vaginal, vulvar, rectal, cervical, and uterine cancers passes dose through structures that determine sexual function and quality of life afterward — the vagina, bulboclitoris, urethra, paraurethral glands, and the neurovascular bundles that serve them. Sparing those structures requires contouring them, and contouring them by hand is slow and inconsistent, because they are small, low-contrast, and tangled together on both CT and MRI. The automated tools that would help do not exist: over 10% of FDA-approved AI segmentation products target prostate cancer, and none explicitly address female pelvic anatomy. The underlying cause is dataset curation. Widely used multi-organ segmentation corpora omit or underrepresent female genito-pelvic structures entirely, so models trained on them never learn to see them.

RoMedFormer is a 3D transformer segmentation foundation model built to close that gap. It was developed at Emory University with collaborators at Memorial Sloan Kettering Cancer Center and the Icahn School of Medicine at Mount Sinai, and released as a preprint in March 2025. Rather than train a single-task network on a handful of annotated pelvic scans, it works through a three-stage curriculum — self-supervised pretraining on unlabeled CT, supervised fine-tuning on large public multi-organ datasets, then task-specific adaptation to the genito-pelvic target — so that a small institutional dataset only has to teach the last step. The design places it alongside other volumetric segmentation foundation models in the catalog such as BrainSegFounder and SAM-Brain3D.

#Key Features

• Rotary positional embeddings in 3D: Absolute positional embeddings become ambiguous when large volumes are processed by sliding window, since the same anatomy appears at different patch offsets. RoPE instead encodes relative distance and orientation directly inside the self-attention computation.
• Fine-grained tokenization: Volumes are tokenized at 8 x 8 x 8 voxels rather than the conventional 16 x 16 x 16, preserving the detail needed to resolve millimetre-scale structures like the paraurethral gland.
• Cross-modality by construction: MRI and CT are trained and tested jointly, letting the encoder exploit anatomy shared between modalities instead of maintaining one network per scanner type.
• Three-stage transfer curriculum: Masked image modelling, then public multi-organ supervision, then genito-pelvic specialization, with weights carried forward between stages so scarce expert annotations go furthest.
• Lightweight decoder: SwiGLU transformer blocks feed a shallow decoder of transposed convolutions, keeping inference cost well below deep U-Net-style alternatives.

#Technical Details

Self-supervised pretraining used masked image modelling over a pooled collection of public CT datasets — FLARE22, HNSCC, RibFrac, ACRIN 6664, and TCIA COVID. Supervised fine-tuning followed on TotalSegmentator, which annotates 104 anatomical structures, and then on the cross-modality AMOS22 dataset containing both CT and MRI. Task-specific fine-tuning used an institutional cohort from the Sexual Toxicity After RT (STAR) study: 30 imaging studies comprising 10 RT planning CT scans at 1.6 x 1.6 x 2 mm and 20 T2-weighted MRI scans at 0.5 x 0.5 x 2 mm, each manually contoured and approved by at least two radiation oncologists.

Training ran inside the nnU-Net framework using the Residual Encoder L preprocessing plan, a patch size of 48 x 320 x 320, five-fold cross-validation, 1,000 epochs of 250 steps, batch size 2, AdamW at learning rate 3e-4 with weight decay 5e-2, DiceCE loss, DropPath 0.2, and Layer Scale initialized at 0.1. Across 11 structures evaluated on 6 CT and 11 MRI scans, RoMedFormer reached a classwise mean Dice of 0.7060 ± 0.0762, mean surface distance 0.5743 mm, and mean centroid distance 3.6893 mm. Per-structure accuracy tracks size and contrast: genitals 0.8677 and bulboclitoris 0.8232 at the top, the vagina hardest at 0.5668, with the inferior hypogastric neurovascular bundles also below 0.66.

#Applications

The immediate use is organ-at-risk delineation during pelvic radiotherapy planning, where automatically generated contours for the bulboclitoris, urethra, vagina, ovaries, and internal pudendal and inferior hypogastric neurovascular bundles let planners write dose constraints for structures that current protocols simply leave uncontoured. The same contours support surgical planning and retrospective dose-toxicity studies linking delivered dose to sexual and pelvic dysfunction outcomes. Because one model covers both T2-weighted MRI and planning CT, it fits clinics whose patients receive either or both.

#Impact

RoMedFormer targets a documented equity gap in medical AI rather than a crowded benchmark, and its multi-stage recipe is a practical template for any anatomy where expert annotation is scarce. The results should be read as preliminary: the preprint has not been peer reviewed, the genito-pelvic cohort is 30 studies from a single institution, and the paper reports RoMedFormer's own metrics without a head-to-head comparison against CNN or transformer segmentation baselines. No code or pretrained weights have been released. The authors describe an ongoing effort to expand the STAR imaging cohort into a public benchmark for genito-pelvic segmentation, which is what the subfield most needs.