RSTAR4D

A linac's on-board cone-beam CT takes about a minute to rotate once around the patient, and in that minute the chest moves. Sorting the projections by a breathing signal into ten respiratory phases recovers that motion, but each phase is then reconstructed from roughly a tenth of the views, clustered rather than evenly spread. The result is a phase-resolved volume buried under streaks — bright and dark bands that can swamp a lung tumor entirely.

RSTAR4D starts from an observation about how those streaks behave. Because the streak pattern follows the projection sampling map, and because breathing is quasi-periodic, the sampling map rotates over the breathing cycle — and so do the artifacts. Optical-flow tracking of the two motions separates them: respiratory deformation is dominated by diaphragm travel along the z-axis, while the rotational streak artifacts deform mainly within the axial plane. A network that sees space and time together can therefore subtract one without erasing the other. Prior deep-learning methods for this task used 2D backbones applied phase by phase, discarding exactly the inter-phase correlation that carries the distinction.

The work comes from Ziheng Deng, Jun Zhao and colleagues in the School of Biomedical Engineering at Shanghai Jiao Tong University, with radiation oncologists and pulmonologists from Shanghai Chest Hospital, and was published in IEEE Transactions on Radiation and Plasma Medical Sciences in 2025 after a March 2024 preprint. It is a sibling of the same group's MARVEL, which attacks cardiac motion in diagnostic CT.

#Key Features

• Separable 4D convolutions: Each 4D filter is decomposed into a 2D convolution in the x-y plane, a 1D convolution along the slice axis, and a 1D convolution along time — a fraction of the parameters an isotropic 3x3x3x3 filter would need, and the reason a 4D network is affordable at all.
• Whole-series inference: The network consumes a complete 256x256x90x10 image (height, width, slice, frame) in a single forward pass on one consumer RTX 3090, rather than stacking independently processed 2D outputs.
• Temporally circular padding: The time-axis convolutions wrap end-exhale back to end-inhale instead of zero-padding, encoding the periodicity of respiration into the architecture.
• Tetris training strategy: Stage I freezes the z-axis convolution branch and trains on abundant 2D+T slices; stage II unfreezes it and finetunes on cropped 4D blocks of three sizes. This is how a 4D model is fit from dozens of 4D volumes without overfitting.
• Lightweight by design: 2.35 million parameters and 13.95 ms per slice at inference, against 113.77 million for the MSD-GAN baseline and 13.69 million for PriorNet.

#Technical Details

The backbone is a U-Net whose 2D filters are replaced by the separable 4D modules; it takes the gated FDK reconstruction concatenated with the all-phase average image as a two-channel input and returns the corrected series. Training pairs were synthesized: 45 4D CT scans from Shanghai Chest Hospital were combined with real breathing signals from the SPARE challenge to simulate Varian On-Board Imager acquisitions at 680 views per one-minute turn, yielding 200 training and 25 validation 4D samples that were then cropped into 18,000 2D+T samples and 3,000 4D blocks. On 11 held-out simulated cases from the 4D-Lung Cancer Imaging Archive, tumor-region SSIM reaches 84.33% against 78.33% for PriorNet, 75.71% for CycN-Net and 31.38% for gated FDK, with global RMSE of 42.54 HU against PriorNet's 49.32 HU. An ablation isolates the two contributions: a 4D model trained conventionally scores 68.20% tumor SSIM, Tetris stage I alone 82.20%, both stages 84.33%.

#Applications

The target setting is image-guided radiation therapy for thoracic tumors, where a legible 4D CBCT acquired at the treatment couch shows how far a lesion travels with the breath and where it sits at each phase. Because the correction runs on projection data the linac already collects during a routine one-minute scan, no protocol change or breath-hold is required. The same argument extends to CBCT-guided percutaneous lung biopsy, where the need for clear images currently pushes clinicians toward general anesthesia for respiratory control.

#Impact

The contribution that generalizes is the demonstration that a genuinely 4D network is tractable in medical imaging at all: separable convolutions make the compute affordable and the Tetris curriculum makes the data requirement survivable, a combination the authors expect to carry to dynamic MR, PET and SPECT. Two limits deserve stating. Training supervision is entirely simulated from 4D CT, so evaluation on the 12 real clinical scans is qualitative plus a projection-domain consistency check (97.02% SSIM, 98.75% NCC) rather than a comparison against ground truth. And the release is thin on terms: author-trained checkpoints and the two training notebooks are committed to the repository, but it carries no license file, leaving reuse rights unstated.