STT-EEG

A clinical EEG study that can afford to label its recordings is usually small: the AD/CN cohort most of this literature trains on holds 65 subjects. Unlabeled resting-state EEG, by contrast, is abundant — every sleep clinic and every neuroscience lab accumulates it. The obstacle to using that surplus is that it was recorded on different amplifiers with different electrode counts, so a model trained on one montage has historically had nothing to say about another.

STT-EEG, developed at Shenzhen University with collaborators in Norway, Pakistan and Saudi Arabia and published in Frontiers in Neuroinformatics in August 2026, is a self-supervised spatio-temporal transformer built around that mismatch. It is pretrained once, by masked reconstruction, on 111 healthy adults recorded on a 64-channel BioSemi system in Oslo, then transferred to a clinical Alzheimer's cohort recorded on 19-channel Nihon Kohden equipment in Thessaloniki. Both corpora are projected down to the 19 electrodes the two montages share and resampled to 250 Hz, so the representation the model carries across is a standard 10–20 clinical one. The authors describe the result as a self-supervised transformer rather than a foundation model, and its pretraining corpus of roughly seven hours is modest against the tens of thousands of hours behind models such as LaBraM, the strongest baseline it is measured against.

#Key Features

• Cross-hardware transfer by construction: pretraining and evaluation use different cohorts, different countries and different amplifiers, so the reported gains reflect generalization across acquisition setups rather than a held-out fold of one recording session.
• Two self-supervised objectives: masked autoencoding reconstructs 75% of hidden signal patches while a temporal-order-prediction head forces the encoder to respect the arrow of time, combined as L_total = L_recon + 0.1·L_temp.
• A spatial attention module: alongside ordinary temporal self-attention, each layer pools over time and computes a softmax weight per electrode, giving the network an explicit, readable channel-importance vector instead of leaving spatial structure implicit.
• Two downstream heads on one backbone: the same pretrained encoder supports binary AD versus cognitively normal classification and continuous Mini-Mental State Examination score regression.
• Interpretability from two independent directions: attention rollout (intrinsic) and leave-one-channel-out perturbation (model-agnostic) are reported together, and agree.

#Technical Details

The encoder is a ViT-style stack of 12 transformer layers with 12 heads and a hidden dimension of 768. Input is a 10-second, 19-channel epoch at 250 Hz — a 19 × 2500 matrix — patched by a 1D convolution with kernel 16 and stride 8 into 311 tokens of width 128, then projected to 768. A hybrid of contiguous-block and random masking hides 75% of patches during pretraining; a lightweight 4-layer, 256-dimensional decoder reconstructs them and is discarded afterwards. Pretraining on the SRM cohort (OpenNeuro ds003775) ran for roughly 48 hours on a single NVIDIA A100 with AdamW and a cosine schedule. Fine-tuning on the Alzheimer's cohort (ds004504, 36 AD and 29 controls) uses subject-stratified 5-fold cross-validation, a reduced backbone learning rate of 1e-5 against 1e-3 for the freshly initialized head, weighted cross-entropy, and same-class Mixup. On ds004504 the model reports 96.42% accuracy and 98.21% AUC-ROC for AD versus control, 93.18% accuracy on the three-way AD/frontotemporal-dementia/control split, and r = 0.872 with 2.34 points MAE for MMSE regression. Ablations attribute +7.08 accuracy points to the self-supervised pretraining, +6.30 to augmentation and +4.86 to the spatial attention module. A separate fine-tune of the same backbone on a third cohort (ds004148, 60 healthy subjects on BrainProducts hardware) reaches 92.18% accuracy.

#Applications

The target use is low-cost dementia screening: resting-state EEG is cheap, non-invasive and already routine in neurology clinics, unlike the PET imaging and CSF assays that anchor Alzheimer's diagnosis. Because the model consumes the 19-channel 10–20 montage that standard clinical amplifiers produce, a clinic with existing EEG hardware and a modest labeled cohort could fine-tune it without new acquisition. The MMSE regression head is the more clinically interesting half — a continuous cognitive estimate read off a passive recording supports longitudinal tracking in a way a binary label does not.

#Impact

STT-EEG's contribution is methodological rather than one of scale: it demonstrates that pretraining on a small cohort of entirely healthy adults yields representations that transfer to disease classification on unrelated hardware, and it quantifies that transfer with an ablation rather than asserting it. The limits are equally clear. The evaluation cohort is 65 subjects from one hospital, so the headline accuracy is a within-cohort cross-validation result rather than external validation; all downstream use is full fine-tuning, with no frozen probe or zero-shot result reported; and no code, weights or checkpoint accompany the paper, whose data availability statement directs readers to the corresponding author. The three EEG corpora are public on OpenNeuro, so the experiments are reproducible in principle, but the released artifact today is the method description.