Loading…
Self-supervised transformer encoding a full eight-hour, seven-channel polysomnogram into task-agnostic representations for automated sleep staging.
No providers recorded yet. Browse all providers
A technologist scoring a polysomnogram assigns a stage to every 30 seconds, but does not decide each epoch in isolation — the surrounding hours and the drift of a patient's own baseline inform the call. Automated stagers have largely not worked that way: they label a 30-second window, or at most a few minutes of context. The longest context window among the models PFTSleep compares against is 90 minutes.
PFTSleep — patch foundational transformer for sleep — takes the whole recording as its input. Eight hours of seven simultaneous channels, resampled to 125 Hz, are cut into 6-second patches and encoded in a single pass, producing a representation of the night that was never optimized for sleep staging at all. A separate, small recurrent head then reads stages off those frozen features. It was developed at the Icahn School of Medicine at Mount Sinai by Benjamin Fox, Ankit Parekh, Girish Nadkarni and colleagues, appearing as a preprint in August 2024 and in SLEEP in March 2025.
That separation distinguishes it from its neighbours. U-Sleep is trained end to end for staging and is very good at it; SleepFM is also self-supervised across brain, cardiac and respiratory channels, but contrastively and over 30-second clips. PFTSleep's argument is that a night-scale, task-agnostic encoder is the more useful object, and that staging is a proof of concept for it rather than the point.
The architecture follows PatchTST with three deliberate departures: patches are 750 samples (6 seconds at 125 Hz) rather than 8 or 16; self-supervision zeroes and augments values within patches rather than masking whole patches; and a key padding mask marks real signal against zero padding. After a learned reversible instance normalization layer, each channel's patches pass through their own linear layer into a 512-dimensional embedding, making the tensor entering the transformer 7 × 4800 × 512. Training used PyTorch Lightning on two H100 80GB GPUs with Optuna for hyperparameter search. Staging is a bidirectional GRU over the frozen features, with probabilities averaged across each group of five patches to yield one label per 30-second epoch.
The corpus was 13,888 studies from SHHS visits 1 and 2, the Wisconsin Sleep Cohort and MrOS visit 1 — 6,818 for training, 2,913 for validation, 4,151 held out — plus 4,169 studies from MESA, APPLES and MrOS visit 2 as independent test sets, 1,011,192 hours of signal in all. Cohen's kappa was 0.81 on the held-out split and 0.78 on WSC, above the 0.72 and 0.75 of prior multi-cohort stagers. On the independent sets it falls to 0.60 (MESA) and 0.59 (APPLES), reaching 0.75 on MrOS visit 2. Retraining the GRU head on MESA — leaving the transformer untouched — lifts MESA to 0.76, while adding MESA to the encoder's pretraining changes almost nothing. On a matched SHHS split, task-specific models such as L-SeqSleepNet and XSleepNet reach 0.83 against PFTSleep's 0.80.
The released package runs inference directly on EDF files, emitting per-epoch stage logits or a HYPJSON hypnogram, which fits how sleep laboratories already store studies. For groups working with National Sleep Research Resource cohorts it offers consistent staging across datasets scored by different centres, and the frozen embeddings are intended as inputs to other sleep questions — incident outcomes, study-specific features — for which no heads exist yet. Its stability from MrOS visit 1 to visit 2, years apart in the same men, suggests longitudinal monitoring.
PFTSleep is the first polysomnography model to encode an entire night in one pass, and its clearest result is a negative one: pretraining the encoder on a new cohort buys almost nothing, while retraining the small classifier head on it buys a great deal. That points at annotation conventions rather than signal distribution as the main obstacle to transfer between sleep centres. The caveats are equally clear — external-cohort agreement is moderate, N1 remains poorly classified as it is for human scorers, the 8-hour input is fixed so longer studies are truncated, and staging is the only downstream task demonstrated so far. Code is Apache-2.0; the checkpoints sit on Hugging Face under a non-commercial licence behind a manual access request.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.