3D-Heart_Seg

Whole-heart segmentation — delineating the four cardiac chambers, the myocardium, and the great vessels in a 3D scan — sits underneath cardiac volumetry, surgical planning, and the anatomical meshes that drive cardiac digital twin simulations. Two problems have held it back. Expert-annotated cardiac volumes are scarce, and a network trained on one imaging modality generally degrades on the other, because contrast in CT and in cardiac MR arises from entirely different physics.

3D-Heart_Seg addresses both by moving the expensive learning off the labeled data. Researchers at the National Heart and Lung Institute at Imperial College London, working with the Alan Turing Institute and University College London, pretrain a 3D encoder on a large corpus of unlabeled cardiac and chest CT and MRI, then fine-tune a decoder on the small labeled splits of public whole-heart benchmarks. Pretraining uses a student-teacher scheme in the style of DINOv2 but built for volumes rather than 2D images: a student encoder is optimized directly while a teacher encoder is updated as a momentum average of the student's weights.

The backbone is an xLSTM-UNet — an encoder-decoder in which the extended LSTM's gating and matrix memory replace the attention or convolution blocks that most 3D medical encoders rely on. That places the model alongside general-purpose 3D medical pretraining approaches such as VoCo and promptable segmenters such as SAM-Med3D, but specialized to cardiac anatomy and to producing dense four-chamber masks rather than prompted regions.

#Key Features

• One model across CT and MR: Multi-modal pretraining and fine-tuning give a single set of weights that segments whole-heart anatomy from either modality, instead of separate modality-specific networks.
• Two rounds of self-supervision: An initial self-supervised stage is followed by a second round on additional unlabeled data, refining the encoder before any labels are used.
• xLSTM encoder: Extended LSTM blocks propagate context across slices of a 3D volume, capturing inter-slice dependencies at lower activation memory than comparable transformer and Mamba encoders in the paper's efficiency analysis.
• Few-label fine-tuning: Only the training splits of the public MMWHS and WHS++ challenges are used for supervision, with the decoder trained under a combined Dice and cross-entropy objective.
• Mesh-ready output: Dense four-chamber masks at high resolution can be converted into 3D heart meshes for computational modeling and simulation-based diagnostics.

#Technical Details

The self-supervised corpus spans cardiac and thoracic imaging: 10,000 UK Biobank cine MR subjects, 31,801 subjects from the NLST chest CT screening trial, 1,204 CT scans from TotalSegmentator, 1,000 patients from ImageCAS, 56 studies from ImageTBAD, CT coronary angiography from the Coronary Atlas, CardioScans cases, and the cine and LGE MR sets released by MICCAI challenges including M&Ms, ACDC, and CMRxMotion. The MMWHS and WHS++ validation splits are also used unlabeled during pretraining, with their training splits reserved for supervised fine-tuning. Optimization uses Adam at a learning rate of 1e-5 over 1,000 epochs, with 80/20 splits and five-fold cross-validation; the reported model carries roughly 30M parameters.

On held-out test data, 3D-Heart_Seg reaches a Dice coefficient of 0.977 ± 0.020 on the WHS++ CT split and 0.931 ± 0.019 on the MMWHS CT split, 0.887 ± 0.052 and 0.871 ± 0.024 on the corresponding MR splits, and 0.771 ± 0.124 on HVSMR-2.0, a congenital heart disease MR benchmark. It records the lowest 95th-percentile Hausdorff distance on all five splits, from 1.251 ± 0.665 on WHS++ CT to 7.757 ± 5.509 on HVSMR-2.0, against CNN, transformer, state-space, and self-supervised baselines including 3D-nnUNet, 3D-UMamba, VSmTrans, and SAM-Med3D.

#Applications

The model targets cardiac imaging workflows where annotation budgets are small and scanner data is mixed. Automated four-chamber masks support chamber volumetry and ejection-fraction estimation, pre-procedural planning in structural heart disease, and the congenital cases represented by HVSMR-2.0. The most demanding downstream use is cardiac digital twins: electromechanical and hemodynamic simulations require patient-specific meshes, and building those by hand from CT or MR is a bottleneck that dense automated segmentation removes.

#Impact

3D-Heart_Seg is a preprint and has not been peer reviewed, and no code or trained weights have been released, so the reported results have not been independently reproduced. Evaluation is retrospective and entirely in-silico, on public challenge data rather than in a prospective clinical setting. Its contribution is a concrete demonstration that xLSTM blocks are a viable backbone for large-scale 3D medical self-supervision — an alternative to the vision transformer and state-space encoders that dominate the area — and that a cardiac-specific foundation model trained across both CT and MR can outperform modality-specific supervised networks on standard whole-heart benchmarks.