S-CEReBrO

Clinical electroencephalography is often continuous. Epilepsy monitoring units, neonatal intensive care, and sleep laboratories record for hours or days at a time, and the diagnostically decisive event may be a single brief discharge buried somewhere in that stream. Transformer-based EEG foundation models are poorly matched to this setting: global self-attention ties the size of the attention state to the length of the signal, so the KV cache grows without bound as a recording continues and eventually overflows available memory. Most EEG foundation models sidestep the problem by operating on short fixed windows, which forfeits the long-range context that continuous monitoring is supposed to provide.

S-CEReBrO (Streaming CEReBrO) attacks this bottleneck directly. Released in July 2026 by Glenn Anta Bucagu, Thorir Mar Ingolfsson, Yawei Li, and Luca Benini at ETH Zurich's PULP Platform, with co-authors at Nanyang Technological University and the University of Bologna, it is an evolution of the earlier CEReBrO architecture built specifically for streaming inference. Its central contribution is Windowed Alternating Attention, which factorizes attention into fixed-size spatiotemporal windows so that only the active window needs resident attention maps. KV cache memory is therefore constant in signal duration rather than linear in it.

The model comes from the same lab and the same BioFoundation codebase as LUNA, and is benchmarked against EEG foundation models including LaBraM and CBraMod as well as BIOT. Where CSBrain and its peers pursue richer tokenization, S-CEReBrO trades global receptive field for a bounded memory footprint — and does so with a backbone of only 2.4M parameters.

#Key Features

• Constant KV-cache memory: Restricting attention to fixed-size windows makes resident memory O(1) in signal duration, so a recording can be extended indefinitely without the attention state growing.
• Windowed Alternating Attention: Windowed Temporal Attention layers model dependencies within each channel; Windowed Spatial Attention layers exchange information across channels at a fixed time index, and the two alternate through the network.
• Staggered receptive fields: Layer-specific dilation and shift factors offset the windows between layers so distant but physiologically related electrodes and time points still interact despite the local restriction.
• Long-context throughput: The model processes signals 100x longer than full self-attention (up to 50,000 seconds, roughly 14 hours) and 3x longer than low-rank linear attention, at 55% of the latter's memory and 2.1x its inference throughput.
• Compact backbone: At 2.4M parameters it reaches state-of-the-art results on 7 of 11 downstream tasks with up to 60% fewer parameters than the foundation models it is compared against.

#Technical Details

S-CEReBrO tokenizes raw EEG with a sliding window and a shared convolutional encoder, preserving topology through an index-based temporal encoding and a continuous spatial embedding produced by an MLP over 3D electrode coordinates. The backbone is three paired WSA/WTA blocks with an embedding dimension of 180 and 5 attention heads, using a spatial window of min(C, 7) and a temporal window of min(T, 5), dilation factors {1, 2, 4}, and shift factors {0, ±1, ±2}, for O(CT) complexity in channels and time. Pretraining uses masked autoencoding over more than 25,000 hours of EEG from more than 12,000 subjects drawn from TUEG, the SEED series, BOAS, SleepEDFx, BCI-NER, and GWD, all resampled to 200 Hz and processed in 30-second windows, with downstream subjects excluded to prevent leakage. Reported results include 89.30 AUROC on TUAB abnormality classification, 87.45 AUROC on CHB-MIT and 85.76 on Neonate seizure detection, 98.23 AUROC on Mumtaz mental-disorder diagnosis, 78.23 weighted F1 on ISRUC sleep staging, and 60.93 weighted F1 on PhysioNet-MI motor imagery.

#Applications

The model targets settings where EEG is recorded continuously and cannot be truncated: long-term epilepsy monitoring, neonatal seizure detection, overnight sleep staging, vigilance and mental-workload estimation, and brain-computer interface decoding. A 2.4M parameter backbone with bounded memory is also well suited to deployment on wearable or edge hardware, the PULP Platform's longstanding focus. The authors state explicitly that S-CEReBrO is a research model and not a medical device.

#Impact

S-CEReBrO reframes an efficiency question that most EEG foundation models have answered by shortening the input: rather than compressing channels or approximating global attention, it bounds the attention state so that duration stops being the binding constraint. Pretrained weights and configuration are published on Hugging Face and the code lives in the Apache-2.0 BioFoundation repository, though the weights themselves carry a CC BY-ND 4.0 license that permits internal fine-tuning but forbids redistributing modified checkpoints, including adapters and quantized variants; official releases are gated through pull requests to the maintainers. The paper is the pre-rebuttal version of a MICCAI 2026 acceptance, with the camera-ready version to follow. The authors note that zero-shot transfer to montages very different from or denser than those seen in pretraining can underperform, and that results vary across cohorts, devices, and labeling protocols.