CAG-MIND

When an interventional cardiologist finishes a coronary angiogram, the report they dictate holds far more than a stenosis grade: it names the dominant vessel, describes the plaque, and records that a balloon and stent were advanced into a particular lesion. Task-specific angiography models throw nearly all of that away, keeping one curated label per study and training a fresh network for each new question. CAG-MIND is built on the opposite premise — that anatomy, lesion morphology and the procedural decision they drove are already written down for every case a catheterization laboratory has ever run, and that this text is enough supervision to learn a transferable representation of the coronary tree.

The model was developed in the Department of Cardiology at Zhongshan Hospital, Fudan University, with Fudan's Artificial Intelligence Innovation and Incubation Institute and the Shanghai Academy of Artificial Intelligence for Science, and released as a medRxiv preprint. It pairs each patient's six standardized angiographic cine projections with the structured semantics of their procedural report and aligns the two through bidirectional contrastive learning — the recipe EchoCLIP applied to echocardiography, carried into invasive imaging, where cine acquisition, multi-view dependency, vessel overlap and contrast dynamics make the visual problem considerably harder.

#Key Features

• Patient-level six-view input: coronary reading depends on complementary projections rather than any single frame, so each case is organized into a fixed six-view framework whose per-view features are fused into one patient-level representation.
• Report-derived weak supervision: a language-model-assisted extraction pipeline converts free-text procedural documentation into structured labels for 11 endpoints, with automated rules that reject reports whose lesion descriptions contradict the procedure performed.
• Cross-modal memory banks: angiographic and report embeddings are held in modality-specific banks that supply cross-batch reference features, stabilizing contrastive alignment at a scale where a single batch offers too few informative negatives.
• Zero-shot endpoint prediction: textual prompts or structured semantic prototypes are scored against angiographic representations in the shared latent space, so a new endpoint needs a phrasing rather than a labeled training set.
• Data-efficient adaptation: fine-tuned on 10% of the labeled internal data, the model still outperforms fully fine-tuned image-only and CLIP-style baselines.

#Technical Details

CAG-MIND has four components: a report semantic encoder, a multi-view angiography encoder with a fusion module, the cross-modal memory module, and the bidirectional contrastive objective. Training used 135,475 paired angiography-report cases comprising 812,850 cine videos — 107,839 training and 23,454 validation cases from Zhongshan Hospital, split chronologically at 2024, and 4,182 external test cases from Shanghai Geriatric Medical Center. Across 11 endpoints spanning structural abnormality detection, plaque assessment and interventional decision prediction, zero-shot mean AUROC was 0.686 internally and 0.745 externally, ranging from 0.567 for prior-stent detection to 0.836 for coronary stenosis in the internal cohort. Supervised fine-tuning raised the mean to 0.827 and 0.846, with stenosis detection at 0.940 in both cohorts, balloon/stent requirement at 0.900 and 0.907, and CABG recommendation at 0.877 and 0.875. CAG-MIND exceeded BiomedCLIP and an angiography-adapted CLIP baseline in the zero-shot setting, and 3DCNN, Xception and Swin3D after fine-tuning. The preprint reports no parameter count and names no pretrained initialization for either encoder.

#Applications

The immediate use is decision support inside the catheterization laboratory: flagging stenosis, calcification, ectasia, plaque rupture and thrombus during or after a diagnostic run, and predicting whether a case will call for balloon or stent placement, rotational atherectomy, thrombus aspiration, intravascular imaging or surgical referral. Because the same pretrained representation serves every endpoint, groups with modest labeled cohorts can adapt it to further downstream questions — culprit lesion identification, SYNTAX scoring, no-reflow risk — without assembling a foundation-scale dataset of their own. The authors position all of this as support for physician judgment rather than a substitute for it.

#Impact

CAG-MIND extends the image-report pretraining paradigm from chest radiographs and echocardiography into invasive coronary imaging, and demonstrates at six-figure patient scale that routine procedural text is a viable supervision signal for a modality whose labels are otherwise expensive. Its constraints are real: no code or weights have been released, the study is retrospective and confined to two Shanghai centers with no multi-vendor or prospective validation, the endpoint labels inherit whatever operator and institutional bias the source reports carry, and a training corpus of routine clinical angiograms cannot be shared. Zero-shot accuracy is modest in absolute terms on several endpoints, and the authors explicitly decline to read the Grad-CAM maps as evidence of causal reasoning. What the work establishes is the feasibility of the approach; whether it changes procedure times, treatment selection or patient outcomes is untested.