BrainGPT

Two EEG recordings of the same mental state rarely look alike to a model. One lab records 62 electrodes at 200 Hz, another 30 at 1000 Hz, a clinical sleep study uses two. A model that consumes a whole montage as a fixed-shape array has to pad, drop or interpolate channels to move between datasets, which is why most EEG deep learning gives up and trains a separate specialist per corpus. BrainGPT shrinks the modeling unit instead: one electrode's time series is one training sample, prefixed by a learned token naming which electrode it came from. Any recording from any headset decomposes into a bag of such samples, so datasets that share no montage collapse into a single homogeneous pretraining set spanning 138 distinct electrode positions.

BrainGPT was developed at the Institute of Automation, Chinese Academy of Sciences, with the University of Chinese Academy of Sciences and Beijing Jiaotong University, and posted to arXiv in October 2024. It was originally named EEGPT and renamed in the August 2025 revision; some result tables in that revision still carry the old row labels. Both names are taken elsewhere — EEGPT is also a masked-autoencoder EEG model from a different group, and BrainGPT also names an unrelated neuroscience-literature language model — so repositories and checkpoints under either name generally belong to something else.

Because the per-electrode decomposition discards spatial layout, a second stage puts it back: a graph whose nodes are electrodes, shared across every downstream task. This separates temporal modeling from spatial modeling rather than entangling them in one attention stack, the design choice that most distinguishes it from masked-reconstruction EEG models such as LaBraM and CBraMod.

#Key Features

• Electrode-wise sampling: Treating each electrode independently makes montage differences invisible to the encoder and lets any subset of the 138 covered positions serve as input.
• Autoregressive pretraining: The encoder predicts the next one-second signal token under a causal mask, minimizing mean squared error on the raw waveform rather than reconstructing masked patches.
• Task-shared electrode graph: A graph attention network activates only the nodes present in a given dataset, so 12 benchmarks train jointly through one shared spatial module.
• Frozen backbone transfer: Downstream adaptation trains only the graph and task heads; the pretrained temporal encoder is never fine-tuned, so adding a task costs a small head.
• Measured scaling behavior: Loss and accuracy were tracked across four model sizes and five training-data volumes, and both curves were still improving at the largest setting.

#Technical Details

The Electrode Temporal Encoder is a decoder-style transformer with multi-head causal attention and Swish feed-forward layers. Signals are resampled to 256 Hz, band-pass filtered between 0.1 and 100 Hz, cut into four-second windows and split into 25 overlapping tokens of 256 samples each, z-scored with no artifact rejection. Four configurations were trained — 1.46M, 11.29M, 183.8M and 1.09B parameters, from 3 to 20 encoder layers and hidden size 128 to 1,792 — each for three epochs over 37.5M electrode samples totaling roughly 1B tokens, on eight NVIDIA A800 GPUs with DeepSpeed ZeRO. Multi-task fine-tuning uses 181K samples over 10 epochs. On 12 datasets grouped into five tasks, the 1.09B model's average accuracy exceeds the best task-specific specialist by 5.07 points on emotion recognition, 6.05 on motor imagery, 8.50 on mental workload, 11.20 on sleep staging and 5.10 on cross-modality decoding, all under cross-subject splits with no subject shared between train and test. Holding architecture fixed and swapping the objective, autoregressive pretraining beat masked reconstruction by more than 2 points on average.

#Applications

The evaluated tasks map onto standard EEG application areas: affective computing on DEAP, FACED and SEED-IV/V; motor-imagery brain-computer interfaces on MIBCI and BCI Competition IV-1; mental workload monitoring on EEGMat and STEW; sleep staging on Sleep-EDF and HMC; and cross-modality decoding on imagined-speech data. Because the temporal encoder stays frozen, a group with a new montage or label set fits only a small graph head rather than a full backbone. Embeddings extracted for DREAMER, a dataset excluded from pretraining, separate by emotion category without additional training.

#Impact

The contribution is methodological rather than a usable tool. It supplies a controlled comparison showing that next-token prediction on raw EEG outperforms masked reconstruction under matched architectures, and scaling curves for autoregressive EEG models, alongside the channel-wise autoregressive pretraining in NeuroLM. It also shows synergy between tasks: joint training beats separate training on all five, most strongly on the smallest dataset. The limitations are substantial. Both the 2024 and the 2025 version state that training code and checkpoints will be released, and neither has appeared, so nothing here is independently reproducible. The work remains a preprint with no journal version, all baseline numbers are the authors' own reproductions, and there is no clinical validation beyond benchmark accuracy.