EEG foundation model that swaps dense attention for gated temporal and spatial paths, pretrained on about 60,000 hours of scalp recordings.
No providers recorded yet. Browse all providers
An EEG recording is a grid: electrodes on one axis, time on the other, and the dependencies that matter mostly run along those axes. Motor imagery is a left/right difference in mu and beta rhythms between sensorimotor electrodes at the same moment; a sleep stage is a rhythm sustained on each electrode over tens of seconds. Dense self-attention treats every electrode–time pair as equally plausible. Under masked-autoencoder pretraining, the authors argue, this invites a shortcut: guess a masked patch from a broad average of everything visible, diluting the localized, lateralized signals downstream tasks depend on.
AXON (AXis-factorized Operator Network), introduced by Mannas AI with a collaborator at IIT Mandi in a September 2026 preprint, removes the dense path entirely. Its Adaptive Anisotropic Attention has a temporal path, where each token attends only to its own electrode across time, and a spatial path, where it attends only to other electrodes at the same time step. A small gate gives each token two non-negative weights summing to one, so every token in every layer picks its own blend. Any two tokens are still connected after two layers, one step along each axis, but direct off-axis interactions are gone by construction.
AXON belongs to Mannas AI's line of EEG encoders alongside MANAS-2 and PRISM; where MANAS-2 changes the pretraining objective, AXON changes the attention operator. It is unrelated to the X-ray-to-CT model AXON.
The encoder has 22 layers of width 512 with 8 attention heads (141.9M parameters), followed during pretraining by a 4-layer dense decoder. EEG is resampled to 200 Hz, mapped to 21 canonical 10–20 electrode positions, and cut into 10-second windows of 11 one-second patches per electrode (231 tokens). Pretraining masks 55% of tokens in spatiotemporal blocks and minimizes an L1 reconstruction loss plus an auxiliary pooled-token reconstruction term, for 50 epochs at batch size 4,096 on 8 NVIDIA H200 GPUs. The pooled, label-free corpus combines the Temple University Hospital EEG Corpus (about 15,000 subjects, 25,000 hours), I-CARE (about 600 post-cardiac-arrest patients, 33,000 hours), and two internal clinical collections totalling 5,589 subjects and 2,981 hours; subject identities are disjoint from every evaluation set.
Evaluation covers six public datasets — motor imagery (PhysioNet MI and BCI Competition IV 2a), cognitive workload (STEW), sleep staging (HMC), seizure detection (Siena) and dementia diagnosis (AD/FTD) — on subject-disjoint splits. AXON reaches mean balanced accuracy of 0.579 with a frozen encoder and 0.656 with full fine-tuning, against 0.550 and 0.606 for an identically trained dense encoder and 0.560 and 0.628 for divided space-time attention. The largest gains are on motor imagery under linear probing (+10.2 points) and dementia under fine-tuning (+10.0 points). Under the same protocol, the released checkpoints of EEGPT, LaBraM, CBraMod, BIOT, CSBrain and REVE score lower, with REVE best at 0.527 and 0.614; the authors note that their own dense baseline already beats all six on frozen probing, so that margin partly reflects their training setup rather than the attention design.
AXON is a general EEG encoder for label-scarce settings: motor-imagery decoding for brain-computer interfaces, cognitive workload estimation, sleep scoring, seizure flagging and dementia screening from resting-state EEG. Because frozen-probe results are strong, a small classification head can be trained on a modest labelled cohort. The design carries to any signal on two meaningful axes; in a separate spectrogram experiment, every factorized variant beat dense attention on AudioSet fine-tuning and SpeechCommands probing.
AXON's contribution is a design rule for structured biosignals: align attention with the signal's own axes and let the model learn how much of each to use. Its ablations — parameter-matched dense attention, divided space-time attention, 2D rotary position embeddings, nearest-neighbour electrode masking and gate interventions — locate most of the gain in a learned depth-dependent schedule rather than in per-token routing. The work is an unreviewed preprint. No code, weights, model card or data card have been released, and part of the pretraining corpus is internal clinical EEG that is not public, so the results cannot yet be reproduced outside the group.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.