CardioFM

Every monitored hospital bed already records the same heartbeat twice. An ECG electrode captures myocardial depolarization and repolarization; the pulse oximeter clipped to the finger captures the photoplethysmogram (PPG), the optical trace of the arterial pulse that beat produces downstream. The lag between them is pulse arrival time, a correlate of blood pressure: the two signals are coupled, not redundant. Cardiac foundation models are nonetheless built one sensor at a time, and stacking the best unimodal encoders after the fact does not recover the coupling — concatenating ECG-FM and PaPaGei embeddings into a 1,280-dimensional feature vector is beaten on ICU false-alarm triage by a 512-dimensional representation learned from both waveforms jointly.

CardioFM is that jointly learned representation. Built by the Kamaleswaran Lab at Duke University with collaborators at Duke University School of Medicine and Emory University, and posted as a preprint in May 2026, it pairs modality-specific 1D convolutional encoders with a bidirectional cross-modal attention module: the ECG branch attends to the PPG branch and the PPG branch attends back, so each representation can draw on the other when its own channel is corrupted by electrosurgical interference, motion, or vasopressor-induced vasoconstriction. A discrete bottleneck built from adaptive residual vector quantization then compresses each fused segment into a short token sequence.

That bottleneck is the design bet. Cardiac physiology is organized around recurring discrete events — sinus beats, ectopic complexes, systolic upstrokes, dicrotic notches — and quantization gives the model a vocabulary of morphological states rather than a diffuse continuous latent space.

#Key Features

• Bidirectional cross-modal fusion: Two attention layers exchange information between the ECG and PPG streams before quantization, rather than concatenating separately trained encoders.
• Adaptive residual vector quantization: Eight residual stages with 64 codewords each produce discrete tokens per 10-second segment, with codebooks generated conditionally on the partial reconstruction instead of held fixed.
• Frozen encoder, lightweight heads: Downstream tasks train only a two-layer MLP on pooled embeddings, so a new clinical endpoint needs little labelled data and no re-pretraining.
• Hardware-agnostic transfer: Zero-shot reconstruction holds across five independent datasets spanning 64–500 Hz sampling, ICU monitors through consumer wearables.
• Edge-sized and hardware-free: The 17.11-million-parameter encoder fits edge inference budgets and reads only signals commodity bedside monitors and consumer wearables already produce.

#Technical Details

Each modality encoder is a 1D CNN stem plus four strided residual blocks that take a 10-second, 250 Hz segment (2,500 samples) down to 40 temporal positions of 256 channels. Two pre-norm cross-modal attention layers with 8 heads fuse the streams, followed by per-modality residual vector quantization with L = 8 stages and K = 64 codewords. Symmetric transposed-convolution decoders reconstruct both waveforms; the pretraining loss is smooth-L1 reconstruction plus quantization commitment terms, with gradients passed through the codebook by straight-through estimation. Pretraining ran 25 epochs on 8 NVIDIA H200 GPUs, over more than 500,000 hours of temporally aligned ECG–PPG from approximately 63,000 patients across several US tertiary care centers plus public ambulatory, laboratory and wearable corpora, split at the patient level. On PTB-XL the model reaches an F1 of 0.86 against 0.79 for a ResNet1D trained from scratch; QT interval regression gives 20.2 ms mean absolute error and pulse arrival time 22.7 ms. Frozen embeddings on the VTaC ventricular-tachycardia false-alarm task score AUROC 0.71 in the multimodal configuration, ahead of ECG-FM at 0.685 with a 90.9 M-parameter encoder and of TimesFM, whose general-purpose forecasting pretraining lands near chance at 0.52.

#Applications

The intended setting is continuous bedside and ambulatory monitoring: automated QT surveillance for patients on QT-prolonging drugs, cuffless hemodynamic trending from pulse arrival time for patients who do not warrant an arterial line, suppression of the false arrhythmia alarms that drive alarm fatigue, and physiologic feature extraction from waveforms hospitals already archive but rarely mine. Because the encoder stays frozen and the tokens are compact, a hospital group can add endpoints incrementally instead of commissioning a separate vendor model per task.

#Impact

CardioFM is an argument that joint pretraining, not scale, is what cardiac biosignal models have been missing — a 17 M-parameter encoder outperforms a 200 M-parameter general time-series model and a 90 M-parameter ECG specialist on the same frozen-feature protocol. The caveats are stated plainly by the authors: pretraining used Lead-II only, so multi-lead configurations need architectural work; inference requires both modalities present, and behaviour under prolonged single-modality dropout is uncharacterized; evaluation is retrospective; and the corpus comes from US academic medical centers, which matters given known PPG signal-quality disparities across skin pigmentation. The work is a preprint awaiting peer review. Source code is promised under an MIT license at publication and has not been posted, and no pretrained weights have been released, so the model is at present a described architecture rather than a downloadable one.