EEGPT

The most predictable part of a scalp EEG recording is usually the part that carries no information: amplifier drift, line noise, muscle artifact, background rhythm. A masked autoencoder trained to reconstruct raw voltage is rewarded for recovering all of it, which is why reconstruction quality and representation quality come apart on low-signal-to-noise biosignals. EEGPT keeps the reconstruction objective but adds a second one that never touches the waveform: the features it predicts for masked patches are aligned against the features a momentum encoder computes from the complete, unmasked signal. That branch compares two representations rather than two waveforms, pushing the encoder toward what stays stable across a recording.

EEGPT was developed in the Faculty of Computing at Harbin Institute of Technology and presented as a poster at NeurIPS 2024. It belongs to the cohort of EEG representation models that includes BENDR, LaBraM and CBraMod, and it takes a strict position on transfer: every downstream number in the paper comes from linear probing, with the encoder frozen and only an adaptive spatial filter and a linear layer trained per task. Because the probe is that thin, accuracy is a property of the representation rather than of per-dataset tuning.

An unrelated EEG model also released as EEGPT in October 2024 — a 1.1B-parameter autoregressive model from a different group — was renamed BrainGPT in a later revision and is cataloged as BrainGPT. The two share no authors and no architecture.

#Key Features

• Dual self-supervised pretraining: An alignment loss matches predicted masked-patch features to a momentum encoder's output, and a reconstruction loss recovers the raw patches; ablating the alignment term degrades downstream performance by 6–9%.
• Local spatio-temporal embedding: Patches carry a learned per-channel embedding drawn from a codex keyed by electrode name, so a dataset recorded on an unfamiliar montage maps onto the model's channel space instead of requiring retraining.
• Summary tokens: Learnable tokens appended to each time segment collect the encoder's output into a fixed-size representation regardless of channel count.
• Robustness to missing channels: Pretraining masks 80% of channel patches alongside 50% of time patches — the same corruption a partial or mismatched electrode set produces at inference.
• Linear-probe transfer protocol: The encoder is frozen for every downstream task, avoiding the overfitting that full fine-tuning invites on small BCI datasets.

#Technical Details

The encoder, predictor and reconstructor are vision-transformer stacks; a momentum encoder of identical structure is updated from the encoder with a factor of 0.01. Input is 58 electrodes at 256 Hz over 4-second windows, chunked into 64-sample (250 ms) patches, and pretraining ran 200 epochs on 8 NVIDIA 3090 GPUs with AdamW on a OneCycle schedule. The corpus mixes five public task datasets — PhysioMI (109 subjects), HGD (14), the Tsinghua SSVEP benchmark (35), SEED (15) and M3CV (106) — spanning motor imagery, motor execution, steady-state visual evoked potentials and emotion.

Eight variants were pretrained, from 0.4M to 101M parameters, and accuracy rose log-linearly with size, summarized by the authors as ACC = (33.6N)^0.029. All downstream results use the large configuration: 8 layers, 512 embedding dimension, 4 summary tokens, with a 25.3M-parameter encoder. On TUEV event-type classification the linear probe reaches 0.6232 balanced accuracy and 0.8187 weighted F1 against 0.5281 and 0.7492 for the fully fine-tuned BIOT baseline, gaps the paper reports as 9.5% and 6.9%. TUAB abnormality detection is level with BIOT at 0.7983 balanced accuracy and 0.8718 AUROC. Elsewhere the probe reaches 0.5846 on BCIC-2A motor imagery, 0.7212 on BCIC-2B, 0.6917 on Sleep-EDFx sleep staging, 0.5837 on KaggleERN error-related negativity and 0.6502 on PhysioP300.

#Applications

The evaluation suite spans the practical range of EEG work: motor-imagery decoding for brain-computer interfaces, P300 and error-related potentials for spellers and adaptive interfaces, sleep staging for clinical scoring, and abnormality and event detection on clinical recordings. Those datasets differ from the pretraining corpus in channel count, sampling rate and paradigm, so a lab can attach a linear head to the released encoder and get a working decoder without the recordings or compute to pretrain — which matters most for BCI groups whose datasets run to tens of subjects.

#Impact

EEGPT is now a standard comparison point for EEG foundation models, cited as a baseline by EEG-JEPA, LaBraM++ and others, and the Braindecode library ships an EEGPT implementation with the pretrained encoder on the HuggingFace Hub. Two limits bound the result. The pretraining corpus is a few hundred subjects across five curated task datasets rather than a clinical archive of the TUEG kind, and the authors close by naming corpus size and model scale as the next steps. And only the large encoder was released, distributed from a figshare share link rather than a versioned archive, so the eight-variant scaling study that motivates the size choice cannot be re-run from released weights.