TT U-Net

A cardiac CT scanner needs on the order of a hundred milliseconds of gantry rotation to collect one image's projections, while a heart at 75 bpm completes a cycle in roughly 800. Rays arriving from different angles record a moving object, and the reconstruction fuses those inconsistent views into blurred walls and streaks trailing off the coronary arteries. ECG gating hides the problem by aiming the exposure at quiescent diastole, but it fails at high or irregular heart rates — and by construction in dynamic cardiac CT, where the point is to image every phase and no phase is quiet.

Removing that blur afterwards runs into a supervision problem: the motion-free version of a beating heart is medically unattainable, so there is no ground truth to train against. TT U-Net answers both halves. Its training corpus, PAD (Pseudo All-phase clinical-Dataset), is synthesized — a 4D statistical motion model derived from XCAT phantoms animates static clinical CT images into a beating sequence, and a simulated cone-beam scan of that sequence supplies the motion-corrupted partner of each clean frame. The network then treats artifact reduction as video deblurring: reconstructions at successive cardiac phases are frames, and self-attention along the temporal axis encodes how each structure moved — the information that explains the artifact.

TT U-Net comes from Ziheng Deng, Jun Zhao and colleagues in the School of Biomedical Engineering at Shanghai Jiao Tong University, published in IEEE Transactions on Medical Imaging in 2023. It is the image-domain sibling of the group's later MARVEL, which attacks the same artifacts inside the reconstruction by compensating the back-projection for a predicted motion field rather than restoring its output.

#Key Features

• Temporal self-attention: At each spatial location the frames of the phase series form a token sequence, letting the network relate a voxel to its own trajectory through the cycle rather than infer motion from a single blurred slice.
• Whole-heart, all-phase correction: Complete volumes are restored across the cycle, not patches sampled along a coronary centerline, so chamber walls and myocardium are corrected alongside the vessels.
• Pure post-processing: Input and output are both reconstructed CT images — no raw projections, scanner geometry, or protocol change — so the model drops onto an existing pipeline.
• Synthetic supervision that transfers: Training data is entirely simulated, yet the released checkpoint is applied unchanged to real clinical scans, evidence that PAD's artifacts stand in for the unobtainable ground truth.

#Technical Details

The backbone is a U-Net with factorized 3D convolutions: 1×3×3 kernels act within each frame, while 5×1×1 kernels mix along time at full resolution. Three encoder stages halve the spatial resolution and double the width from a base of 24 channels. At each deeper stage a patch-embedding convolution reduces resolution again, two transformer blocks apply temporal self-attention with a learned relative-position bias, and the result rejoins the convolutional features before crossing the skip connection. A global residual connection makes the network predict a correction to the input rather than the corrected image. The transformer block is adapted from SwinIR's windowed attention with the spatial window collapsed to one element, so the attended sequence is time; no SwinIR weights are used, and training is from scratch.

Optimization combines an L1 loss with a WGAN-GP adversarial term against a PatchGAN discriminator, weighted at 0.001, which discourages the over-smoothed textures a pixel-wise loss alone produces. The released implementation fixes the clip at 48 frames of 256×256, needing about 38 GB of GPU memory to train on an NVIDIA A100 over 20 epochs of Adam at a learning rate of 1e-4. PAD's motion model is built by PCA over 21 4D XCAT phantoms of 20 frames each, separating inter-subject anatomy variation from inter-phase motion; the realistic half comes from single-phase clinical cardiac CT, for which the authors point users at the MMWHS and ImageCAS collections.

#Applications

The direct use is salvaging diagnostic quality in the scans gating cannot rescue — patients with tachycardia or arrhythmia, where the chosen reconstruction phase is not actually still. Because the correction applies to whole volumes at every phase, it also supports functional reading of dynamic cardiac CT, where wall motion across the cycle is the measurement rather than a byproduct. PAD is separately useful: with access to the XCAT phantom, other groups can follow the recipe to build paired training data from their own single-phase archives.

#Impact

TT U-Net's durable contribution is the reframing: once a cardiac CT study is read as a video rather than a stack of independent images, the temporal axis supplies the constraint that separates artifact from anatomy, and attention is a natural way to exploit it — a view later cardiac motion-correction work has continued to build on. The release is modest in packaging: one demonstration Jupyter notebook plus MATLAB scripts for the PAD pipeline, with the checkpoint and a few evaluation volumes on a personal cloud drive and no license attached to either. PAD cannot be redistributed at all — the authors have no permission to share the XCAT phantom behind it — so the dataset is a procedure rather than a corpus.