EEGDM

Scalp EEG is strongly autocorrelated in time. Blank out half a second of a channel and the surrounding samples usually pin down what was removed — so a model trained to recover masked segments can get very good at local interpolation without ever representing the global dynamics that make a recording diagnostic. That objective is what nearly every recent EEG foundation model uses, LaBraM and CBraMod among them, and it is the gap EEGDM was built to close.

EEGDM replaces masked reconstruction with generation. A conditional latent diffusion model is asked to produce a complete multi-channel EEG segment from pure Gaussian noise, and a separate encoder supplies the only information it gets about which segment to produce. The conditioning vector is not detached during pretraining, so the denoising loss back-propagates through it and the encoder is pushed to compress whatever is actually needed to regenerate a recording — cross-channel structure, phase relationships, long-range temporal pattern — rather than whatever is needed to fill a gap. Once pretraining ends, the diffusion module is set aside and the encoder alone carries into downstream tasks.

The model comes from the MOE-Key Laboratory of Pervasive Computing at Tsinghua University, with a collaborator at Tianjin University, and was first posted in August 2025. An unrelated 2025 preprint by Puah et al. is also called EEGDM; that one builds a structured state-space model into the diffusion process, and the authors here call it "Puah-EEGDM-SSM" to keep the two apart.

#Key Features

• Generative pretraining objective: The encoder is trained through a denoising loss on whole signals rather than a reconstruction loss on hidden patches, forcing it to carry holistic temporal and cross-channel structure.
• Diffusion in a PCA subspace: Denoising runs on a 20-component PCA projection of each EEG patch instead of the raw high-dimensional signal, so the diffusion model can focus on dominant signal structure and the encoder is left to supply semantics.
• Channel augmentation as conditioning: Each sample is encoded alongside two semantics-preserving augmentations — a 0.4-second per-channel zero-mask and an amplitude rescale — and the three representations are averaged into the conditioning vector.
• One checkpoint, six downstream sets: A single TUEG-pretrained encoder is applied to six datasets it never saw in pretraining, with frozen-encoder linear probing reported alongside full fine-tuning.
• Synthetic EEG for augmentation: The diffusion module remains usable as a generator; sampling extra CL-Drive training data lifted balanced accuracy from 53.57% to 58.96%.

#Technical Details

The encoder is a Vision Transformer variant: a single 1-D convolution (512 channels, kernel and stride both 200) tokenizes one-second EEG patches, and an 8-block transformer with hidden dimension 512, 8 attention heads and a 2048-wide MLP embeds them, average-pooled to a single vector. The denoiser is a matched stack of 8 Diffusion Transformer blocks using adaLN-Zero modulation over 1,000 timesteps. The downstream network, encoder plus linear predictor, is about 25M parameters.

Pretraining ran from random initialization on the Temple University Hospital EEG Corpus, 19 channels resampled to 200 Hz and cut into 30-second segments, for roughly 329,400 steps on four A100 GPUs (about 113 wall-clock hours). On TUEV, a six-way clinical event task with class imbalance reaching 80:1, the fine-tuned model reaches 0.6826 balanced accuracy and 0.9169 macro AUC-ROC, against 0.6424 and 0.8976 for CBraMod; the same encoder scores 0.6010 under linear probing and 0.5825 trained from scratch, isolating what pretraining contributes. Its SPSW minority-class recall of 50.3% contrasts with the 10–18% of the masked-reconstruction baselines. On TUAB the model reaches 81.80% balanced accuracy and 90.85% AUC-PR. A separate same-distribution ablation pretrained directly on TUEV reaches 0.7971 balanced accuracy, reported to compare against Puah-EEGDM-SSM under matched conditions rather than as the released artifact.

#Applications

The encoder suits groups with a labeled EEG set too small to train a decoder from scratch: clinical event and abnormality screening, motor-imagery brain-computer interfaces (SHU-MI), driver cognitive-load estimation (CL-Drive), and categorical and distributional emotion recognition (Seed-VII, DMER). The retained diffusion module gives a second, less common option — synthesizing extra labeled trials to rebalance a skewed training set before fine-tuning.

#Impact

EEGDM shows that a generative objective is a viable alternative to masked modeling for EEG representation learning, and its most useful result is not a leaderboard position but the minority-class recall: on a task where the rare classes are the clinically interesting ones, a generative pretext holds recall that masked pretraining loses. The work remains a preprint and has not been peer reviewed, and it does not lead on every metric: CBraMod stays ahead on TUAB and on the sample-weighted TUEV metrics. Downstream datasets were chosen to favor the last five years, with TUAB and TUEV kept as legacy anchors, so comparability with older EEG benchmarks is deliberately narrow. Code and all four pretrained checkpoints are public.