FAME

EEG foundation models have a scaling problem: adding pretraining data or parameters often fails to improve downstream decoding. FAME — Frequency-balanced Autoencoding for Masked EEG — argues that the bottleneck is not capacity but what the pretraining objective actually rewards. Probing the representations of existing pretrained EEG encoders, its authors find a systematic low-frequency preference: broadband embeddings align far more closely with low-frequency band-limited embeddings than with high-frequency ones, and activity above roughly 30 Hz is measurably harder to decode back out of a frozen representation. The bias persists across architectures, pretraining objectives, and dataset scales, and grows rather than shrinks with model size.

Two effects compound to produce it. Scalp EEG power follows an approximate 1/f power law, so slow rhythms dominate the signal's energy; separately, neural networks fit low-frequency structure before high-frequency structure during optimization. Masked autoencoding adds a third, objective-level source. By Parseval's theorem, a pointwise squared error in time equals a sum of squared spectral errors weighted by signal power, so equal relative error costs far more at a high-power delta component than at a low-power gamma one. A model can therefore drive its reconstruction loss down on slow, high-energy activity while receiving almost no gradient for the fast dynamics that carry motor imagery, sleep spindles, and high-frequency oscillations in epilepsy.

FAME's correction changes the reconstruction target rather than the architecture. Instead of predicting the raw waveform, it predicts log-power time-frequency activity in predefined EEG bands, standardizes each band independently, and weights every band's loss equally, decoupling supervision strength from a band's intrinsic power. It was released as a preprint in August 2026 by a team led by Southern University of Science and Technology's Department of Biomedical Engineering with Omni-Intelligence and academic collaborators.

#Key Features

• Band-standardized reconstruction targets: Each frequency band's targets are normalized with corpus-level statistics before the loss is computed, removing the orders-of-magnitude power gap between delta and gamma.
• Equal band weighting: Band-wise losses are averaged with uniform weight, so neither high-power bands nor bands containing more frequency bins can dominate the objective.
• Dense decoder supervision: Masking corrupts only the encoder input; the decoder predicts every channel-time-band target, masked and unmasked alike.
• Configurable spectral partition: The band split is a hyperparameter — five canonical bands for large-scale pretraining, nine uniform 5 Hz bands for analysis.
• Quantitative bias diagnostics: Three metrics (low-frequency bias, frequency imbalance, signed frequency slope) score any encoder's spectral balance from band-wise decoding losses.

#Technical Details

FAME splits an EEG segment into non-overlapping temporal patches, replaces a random subset with mask tokens, and encodes them with a Transformer, trained at 50M and 1B parameters with 50% and 75% masking respectively. Targets are log-power time-frequency coefficients averaged within the delta, theta, alpha, beta, and gamma bands, then standardized per band. Pretraining used twelve public EEG datasets spanning visual, auditory, somatosensory, motor, cognitive-state, and decision-making paradigms, and took about 12 hours on eight NVIDIA H20 GPUs. Evaluation is by linear probing on a frozen encoder across 41 downstream tasks in OmniEEG-Bench, against BIOT, BrainOmni, CBraMod, EEGMamba, FEMBA, LaBraM, NeuroGPT, NeuroLM, and REVE. FAME-50M is best on 14 of the 41 tasks and FAME-1B on 24, with the largest margins on ADHD classification (86.4% balanced accuracy versus 71.1% for the strongest baseline), TUEV event classification (80.7%), and high-frequency-oscillation detection (79.2%). Under full fine-tuning on the 13 EEG tasks of NeuralBench, FAME-50M records the best average balanced accuracy at 61.4%, ahead of REVE at 60.9% and LaBraM at 59.8%.

#Applications

The frozen-encoder-plus-linear-probe protocol is the practical mode of use: one pretrained checkpoint supplies features for clinical classification (neurodegenerative disease, ADHD, depression, sedation depth), sleep staging, epileptiform event and high-frequency-oscillation detection, motor imagery and SSVEP brain-computer interfaces, and affective state decoding — without retraining the backbone per cohort, which matters for the small, montage-heterogeneous datasets typical of clinical EEG. The three bias metrics are separately useful for screening candidate encoders before committing to a downstream pipeline.

#Impact

FAME reframes a field centred on tokenization and architecture — the axes explored by CSBrain and LUNA — as a question about the pretraining target, showing that the reconstruction loss itself encodes a spectral prior no amount of scale will undo. The authors are careful about the limits: frequency balance helps most on tasks drawing on broad spectral information and can be neutral where slow activity genuinely carries the signal, and the bias-versus-performance correlations rest on too few pretrained models to support strong inference. Computing and storing time-frequency targets adds preprocessing cost, and the fixed equal-weight partition is a task-agnostic default rather than an optimum. The work is a preprint awaiting peer review, and neither model weights nor training code have been released; the companion OmniEEG-Bench repository is the evaluation harness, not the model, and carries no declared license.