SleepGPT

SleepGPT is a sleep-specific language model that treats an overnight hypnogram — the sequence of sleep stages scored across a night of sleep — as a "language" and learns its grammar. Rather than classifying raw polysomnography (PSG) signals directly, SleepGPT models the sequential dynamics of sleep macrostructure, capturing the characteristic transitions among wake, N1, N2, N3, and REM that define healthy and pathological sleep. This lets it act as a general-purpose module that both corrects the output of existing sleep-staging models and supplies discriminative features for sleep disorder diagnosis.

Developed by researchers at South China University of Technology with clinical collaborators (the preprint was first posted to medRxiv in October 2024, initially under the name HypnoGPT), SleepGPT addresses a persistent weakness of signal-level stagers: they classify each 30-second epoch largely in isolation and ignore the long-range temporal context that human scorers rely on. By learning that context explicitly from millions of expert-annotated stages, SleepGPT provides a lightweight, model-agnostic correction layer that improves accuracy without retraining the underlying classifier.

The model sits at the intersection of biosignal analysis and the foundation-model paradigm, demonstrating that the self-supervised, next-token pretraining recipe that powers natural language models transfers cleanly to clinical time-series when the series is discretized into a small symbolic vocabulary.

#Key Features

• Hypnogram-as-language modeling: SleepGPT operates on discrete sleep-stage sequences rather than raw EEG, learning transition patterns that encode sleep architecture and its disruptions.
• Model-agnostic add-on: It acts as a plug-in correction module that consistently improves the predictions of multiple state-of-the-art signal-level sleep stagers without modifying or retraining them.
• Sleep disorder diagnosis: A hierarchical transformer network using SleepGPT as a local feature extractor classifies abnormal sleep-stage sequences and diagnoses Type-1 narcolepsy directly from hypnograms.
• Wearable EEG support: SleepGPT-enhanced staging maintains high accuracy on low-density wearable EEG recordings, supporting at-home and ambulatory sleep monitoring.
• Self-supervised pretraining: A next-stage prediction objective on large unlabeled hypnogram corpora removes the need for additional manual annotation when adapting to new downstream tasks.

#Technical Details

SleepGPT adopts a GPT-2-style decoder-only transformer with masked multi-head self-attention. Overlapping blocks of K consecutive sleep stages are extracted from each overnight sequence with a stride of one, and the model is pretrained self-supervised to predict the next block of stages. Pretraining uses the Sleep Heart Health Study (SHHS) cohort — roughly 5,793 whole-night recordings containing on the order of 5.8 million expert-scored stage annotations. For disorder diagnosis, SleepGPT serves as the local feature extractor inside a hierarchical transformer network. The model was evaluated as a correction layer on the SleepEDF and MASS staging datasets and validated on independent cohorts including Physio2018, BOAS (simultaneous wearable EEG and PSG), CAP, ISRUC, and the MNC and HANG7 narcolepsy datasets, where it improved staging across multiple base models and surpassed prior approaches on narcolepsy classification. Code is released under the Apache-2.0 license; no pretrained weights are distributed on a public model hub.

#Applications

SleepGPT targets clinical and research sleep medicine. As a correction module it can be attached to existing automated scoring pipelines to raise agreement with expert scorers, reducing the manual review burden in sleep laboratories. Its ability to operate on wearable, low-density EEG makes it suitable for at-home sleep assessment and large-scale epidemiological screening. Beyond staging, its hypnogram-level disorder classifier offers a non-invasive route to flagging conditions such as Type-1 narcolepsy from routine overnight recordings, benefiting sleep clinicians, neurologists, and researchers studying sleep architecture.

#Impact

SleepGPT illustrates that foundation-model pretraining can be applied to the symbolic output of clinical biosignal pipelines, not just to raw waveforms, yielding a portable module that improves heterogeneous downstream systems. By decoupling temporal-context modeling from signal-level classification, it offers an inexpensive accuracy gain that any existing stager can adopt. As a preprint with open Apache-2.0 code, its primary limitations are that benchmark numbers are not yet peer-reviewed and that pretrained weights are not distributed through a dedicated model hub, so reproduction currently requires running the released training pipeline.