PASTA

Radiology foundation models have largely been trained with self-supervised objectives on unlabelled CT volumes. That works well for learning anatomy, but tumours occupy a tiny fraction of a scan, and objectives that reward reconstructing global structure tend not to sharpen the lesion-specific signal oncology depends on. Supervised pretraining fixes the incentive but runs into a supply problem: pixel-level lesion annotations are expensive, many tumour types are rare, and privacy restrictions keep the largest annotated cohorts inside single institutions.

PASTA, from Shanghai Jiao Tong University and Shanghai AI Laboratory with The First Hospital of China Medical University and Pranav Rajpurkar's group at Harvard Medical School, sidesteps that supply problem by manufacturing the supervision. Its companion framework PASTA-Gen synthesises lesions into anatomically healthy template scans under radiologist-derived rules, producing paired 3D CT volumes, pixel-level masks, and structured lesion descriptions. Because every lesion is placed by the generator, the mask and the report are exact by construction rather than annotated after the fact.

That pipeline yielded PASTA-Gen-30K, 30,000 synthetic image-mask-text triples covering 15 lesion categories across ten organ systems, on which PASTA is pretrained. The resulting encoder is then fine-tuned with lightweight task heads for screening, segmentation, structured reporting, staging, survival prediction, and cross-modality transfer.

#Key Features

• Rule-guided lesion synthesis: PASTA-Gen parameterises each lesion by eight clinically meaningful attributes — enhancement, size, morphology, density, heterogeneity, boundary, surface, and invasion — sampled from distributions fitted to real reference lesions, then refined by a 3D denoising diffusion model to suppress artefacts.
• Exact supervision at scale: Every synthetic scan arrives with a pixel-level mask and a matching structured report, giving lesion-level supervision at a scale real annotation cannot reach.
• Label-efficient transfer: Under few-shot fine-tuning with as few as one to sixteen labelled cases and 2,000 iterations, the model retains strong segmentation accuracy, reaching 0.608 Dice for gallbladder cancer from two labelled examples.
• Cross-modality generalisation: Pretrained only on CT, the encoder transfers to MRI, reaching 0.504 Dice on brain tumour segmentation at 16 labelled cases and 0.603 on liver tumours.
• Workflow-integrated deployment: PASTA-AID wraps the fine-tuned models in an interface where radiologists review and edit generated masks and draft reports, rather than consuming raw model output.

#Technical Details

PASTA is a 3D U-Net encoder-decoder with a multilayer perceptron classification head, pretrained in two stages on PASTA-Gen-30K using lesion segmentation and vision-language alignment objectives; downstream tasks reuse the frozen-architecture encoder with task-specific heads under five-fold cross-validation. Synthetic data quality was audited by four radiologists in a blinded Turing-style comparison against real scans, with realism ratings of 4.23-4.73 and image-text consistency of 4.54-4.93 on five-point scales.

On 3,864 non-contrast CT scans from liver, pancreatic, and kidney cancer cohorts, PASTA reaches AUC of 0.964-0.987, ahead of Models Genesis and SuPreM, with the largest margin on liver detection at +0.095 AUC over the next-best model. Full-data lesion segmentation across 1,535 scans and 15 lesion types gives Dice of 0.433-0.814, exceeding nnU-Net, Universal, and the pretrained baselines on all but gallstones. Staging AUC runs 0.738-0.855 across gastric, rectal, and bladder cancers, and survival prediction 0.660-0.878 across lung, gastric, rectal, and bladder cancers. In the simulated reader study with two junior and two senior radiologists under fixed 30-minute sessions, PASTA-AID raised case throughput by 11.1-25.1%, lifted pancreatic tumour recall from 61.3% to 92.7%, cut per-case segmentation time by up to 78.2%, and reduced report preparation time by 15.7-36.5%.

#Applications

The intended setting is oncology radiology at the point of reading: flagging suspicious lesions on non-contrast screening CT, pre-segmenting lesions for volumetric follow-up, and drafting the structured report a radiologist then edits. Its label efficiency matters most for rare tumours, where a department may have only a handful of annotated cases; a few examples are enough to adapt the encoder. The reader study also found that assistance narrowed the gap between junior and senior readers, suggesting value in settings with limited subspecialist coverage.

#Impact

PASTA is a concrete demonstration that generated data with exact labels can substitute for scarce, privacy-encumbered annotation in 3D medical imaging, and it is unusual among imaging foundation models in pairing benchmark results with a registered, workflow-embedded reader study. Code, pretrained weights, and the full PASTA-Gen-30K dataset are public under an MIT license. The clinical evidence remains retrospective and simulated rather than prospective, the reader study draws its cases from a single institution and three cancer types, and the synthetic lesions inherit whatever assumptions the rule-based attribute model encodes.