CREMA

CREMA (Contrastive Regularized Masked Autoencoder) is a self-supervised foundation model for 12-lead electrocardiograms (ECGs), developed by Medical AI Co., Ltd. in Seoul, South Korea, and introduced in a preprint first posted in June 2024. It targets a persistent obstacle in clinical ECG analysis: models trained on one population or care setting often degrade sharply when applied to data from a different hospital, patient mix, or acquisition device. CREMA aims to learn ECG representations that transfer robustly across these distribution shifts, so that a single pretrained backbone can support many downstream diagnostic tasks without bespoke retraining for each site.

The core idea is to merge two complementary self-supervised signals that are usually used in isolation. A masked autoencoder (MAE) objective forces the model to reconstruct deliberately hidden segments of the ECG waveform, encouraging it to internalize the morphology of beats and rhythms. A contrastive regularization term, applied on top of the MAE, shapes the latent space so that semantically similar recordings stay close together while dissimilar ones are pushed apart. The authors frame this hybrid objective as a Contrastive Regularized MAE loss.

CREMA sits within the recent wave of ECG foundation models that adapt the self-supervised pretraining recipes of vision and language to physiological time series. Its distinguishing emphasis is robustness under clinical distribution shift rather than performance on a single curated benchmark.

#Key Features

• Hybrid self-supervised objective: Combines masked-signal reconstruction with a contrastive regularization term, blending generative and contrastive learning so representations capture both fine waveform detail and discriminative structure.
• Signal Transformer backbone: Uses a transformer architecture adapted for physiological signals to model both local waveform morphology and long-range temporal dependencies across the cardiac cycle.
• 12-lead coverage: Operates on full 12-lead ECGs, the standard diagnostic acquisition used across emergency, inpatient, and outpatient cardiology.
• Robustness to distribution shift: Explicitly evaluated on distribution-shifted clinical data to test whether learned representations hold up across heterogeneous care settings rather than only on in-distribution test splits.
• Flexible downstream use: Supports both linear probing of frozen features and full fine-tuning, letting users trade off compute and labeled-data requirements against task accuracy.

#Technical Details

CREMA pretrains a Signal Transformer encoder on 12-lead ECG recordings using the Contrastive Regularized MAE objective: portions of the input signal are masked and reconstructed, while a contrastive term regularizes the embedding space. After pretraining, the encoder is evaluated under two standard protocols. In linear probing, the backbone is frozen and only a lightweight classifier head is trained, isolating the quality of the learned representations. In fine-tuning, the full network is adapted end-to-end to each task. Across these protocols the authors report that CREMA outperforms supervised baselines and prior self-supervised ECG models, with the advantage most pronounced on distribution-shifted clinical data where conventional models tend to lose accuracy. The preprint emphasizes the qualitative robustness gains across clinical domains; readers should consult the paper for the exact datasets, cohort sizes, and per-task metrics.

#Applications

CREMA is intended as a reusable backbone for ECG-based diagnostic tasks such as detecting arrhythmias, conduction abnormalities, and other cardiac conditions from standard 12-lead recordings. Because it is pretrained once and adapted via probing or fine-tuning, it is well suited to clinical settings where labeled data are scarce or where a model must generalize across hospitals and devices. The underlying technology is deployed in production within Medical AI Co.'s AiTiA clinical decision-support platform, which the company reports is used across 50-plus hospitals, illustrating the model family's path from research into operational cardiology workflows.

#Impact

CREMA contributes to the growing effort to build transferable foundation models for physiological signals, and its specific focus on robustness under clinical distribution shift addresses a failure mode that limits the real-world deployment of many ECG classifiers. Its integration into a fielded clinical platform serving dozens of hospitals signals practical relevance beyond benchmark leaderboards. A key limitation for the research community is openness: the model weights and training code are not publicly released, as the work is commercial, so independent reproduction and external benchmarking are constrained and the published preprint is the primary source for its claims.