EEG foundation model built on bidirectional Mamba blocks that scale linearly with recording length, pretrained on 21,000 hours of clinical EEG.
No providers recorded yet. Browse all providers
Clinical EEG is long. An epilepsy monitoring admission produces days of continuous multichannel signal, and even routine studies run twenty minutes at hundreds of hertz per electrode. Transformer-based EEG foundation models such as LaBraM and EEGFormer handle this by windowing, because self-attention's compute and memory both grow with the square of sequence length. That scaling is also what keeps these models off the wearable and bedside hardware where long-term monitoring would be most useful.
FEMBA — Foundational EEG Mamba + Bidirectional Architecture — from the Integrated Systems Laboratory at ETH Zurich with Luca Benini's group at the University of Bologna, replaces attention with a selective state-space model. Mamba processes a sequence with a recurrent scan whose cost is linear in length, so doubling the recording doubles the work rather than quadrupling it. Because EEG interpretation depends on context on both sides of an event, each block runs the scan forward and backward over the signal and sums the two passes.
FEMBA is the founding member of the pulp-bio BioFoundation family of efficiency-oriented biosignal models, later joined by LUNA and S-CEReBrO. Its design goal is explicit: match transformer accuracy on the standard EEG benchmarks at a fraction of the FLOPs and memory, so that the same representation can serve a hospital workstation and a battery-powered device.
Pretraining uses the Temple University Hospital EEG Corpus — roughly 21,000 hours of recordings from more than 5,000 participants — after quartile-based per-channel normalisation against the interquartile range. Patches are projected by a 2D convolution and given learnable positional embeddings before masking. For fine-tuning the decoder is discarded and the encoder feeds one of two heads: a small fully connected stack with GELU activations at about 0.5M parameters, or a Mamba-enhanced head adding one further block at up to 0.7M parameters. All layers are fine-tuned end to end with Adam at a 1e-4 initial learning rate under cosine decay, with layer-wise learning-rate decay of 0.75 during pretraining.
Evaluation spans three TUH subsets: TUAB for binary normal-versus-abnormal classification, TUAR for artifact recognition in four labelling schemes from binary to 13-way per-channel, and TUSL for four-class slowing, seizure, complex, and normal classification. The Huge variant reaches 81.82% balanced accuracy and 0.8921 AUROC on TUAB, within 0.7 percentage points of the best reported transformer result while using 3.5× fewer operations and 1.5× less memory. On TUAR the Base model reaches 0.949 AUROC in the binary setting, and the Tiny model exceeds the previous best result with over 27× less computation and more than 2× better memory efficiency.
The immediate uses are the ones the benchmarks encode: triaging routine EEG as normal or abnormal to prioritise neurologist review, flagging eye-blink, muscle, and electrode artifacts before downstream analysis, and identifying slowing patterns associated with encephalopathy or focal lesions. The efficiency argument extends the reach beyond the reading room — the Tiny variant is small enough for ambulatory monitors and wearable headsets, where continuous on-device inference is the point and offloading raw EEG to a server is not practical.
FEMBA is a direct test of whether attention is necessary for EEG foundation models, and its answer is that a linear-time bidirectional state-space encoder matches or exceeds transformer baselines on the standard benchmarks at substantially lower cost. It established the pulp-bio BioFoundation line, whose later members carry the same efficiency-first premise into continuous monitoring and long-context modelling. Pretrained checkpoints for the Tiny, Base, and Large variants are published on HuggingFace with per-task weights, though under a CC BY-ND license that forbids distributing modified versions. Coverage is confined to the Temple University corpora and to classification tasks; the model has not been evaluated on non-clinical montages, wearable-grade electrodes, or regression targets.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.