CBraMod

CBraMod (Criss-Cross Brain Foundation Model) is a self-supervised foundation model for decoding electroencephalography (EEG) signals across a wide range of brain-computer interface (BCI) tasks. Developed by Jiquan Wang, Sha Zhao, and colleagues at Zhejiang University and presented at ICLR 2025, it targets a long-standing obstacle in EEG modeling: the heterogeneity of recording setups. EEG datasets differ in the number and placement of electrodes, sampling rates, and recording durations, which makes it hard to train a single model that transfers across studies.

The model's central insight is that the spatial dependencies between electrodes and the temporal dependencies along the signal are fundamentally different in nature and should not be modeled with a single, undifferentiated attention mechanism. Prior EEG transformers tend to flatten all patches into one sequence and apply standard self-attention, conflating these two axes. CBraMod instead factorizes attention into parallel spatial and temporal branches, letting the network learn electrode-to-electrode and time-to-time relationships separately before combining them.

Pretrained by masked reconstruction on a large EEG corpus and then fine-tuned on downstream datasets, CBraMod is released as open code under an MIT license with pretrained weights on Hugging Face, making it a practical backbone for both research and applied BCI development.

#Key Features

• Criss-cross attention: A transformer backbone with two parallel attention mechanisms that separately model spatial (across-electrode) and temporal (across-time) dependencies, reflecting the heterogeneous structure of EEG.
• Asymmetric conditional positional encoding: A flexible positional scheme that adapts to diverse EEG formats, so the same model can ingest recordings with different channel counts and durations without redesign.
• Masked-reconstruction pretraining: EEG is split into patches and the model is trained to reconstruct masked patches, a self-supervised objective that learns transferable representations without task labels.
• Broad task coverage: A single pretrained backbone fine-tunes to state-of- the-art results on up to 10 distinct downstream BCI tasks spanning 12 public datasets, demonstrating strong generalization.
• Open release: Code, pretraining and fine-tuning scripts, and pretrained weights are publicly available, lowering the barrier to reproducing and extending the work.

#Technical Details

CBraMod tokenizes EEG by dividing each channel's signal into fixed-length patches; the released configuration processes signals at 200 Hz with 200 samples (one second) per patch, arranged as a channel-by-time grid of tokens (for example, a 22-channel, 4-second input forms a 22 x 4 patch grid). Each transformer layer applies its spatial and temporal attention branches in parallel over this grid, and the asymmetric conditional positional encoding supplies position information that generalizes across channel layouts and sequence lengths. During pretraining, a subset of patches is masked and the model reconstructs them, learning representations directly from raw EEG without manual annotation. The pretrained encoder is then attached to lightweight task-specific heads and fine-tuned on each downstream dataset. Across 10 tasks and 12 datasets, CBraMod reports state-of-the-art performance relative to prior EEG foundation models such as LaBraM and BIOT.

#Applications

CBraMod serves as a general-purpose feature extractor for EEG-based brain- computer interfaces and clinical neurophysiology. Reported downstream tasks include emotion recognition, motor imagery classification, sleep-stage scoring, seizure and abnormal-EEG detection, event-type and mental-state classification, and related decoding problems. Because the same pretrained backbone transfers across these heterogeneous datasets, it is especially useful for groups working with limited labeled EEG, who can fine-tune a strong foundation model rather than training from scratch—benefiting BCI researchers, sleep and epilepsy investigators, and developers of neurotechnology applications.

#Impact

CBraMod advances the case that EEG, like protein sequences and natural images, benefits from large-scale self-supervised foundation models, and its factorized criss-cross attention offers a concrete architectural lesson: spatial and temporal structure in biosignals are better modeled separately than jointly. Its acceptance at ICLR 2025, permissive MIT licensing, and publicly released weights have made it a visible reference point and reusable backbone in the fast-growing EEG foundation-model literature. As with any model pretrained on a particular corpus of recording montages, transfer to substantially different electrode configurations or atypical clinical populations should be validated empirically before deployment.