Menstrual Foundation Model

Someone logs cramps on a Tuesday, nothing for three weeks, then a period and four mood entries in two days. That record is simultaneously about her physiology and about her attention: the gaps encode logging habits as much as the absence of symptoms. Tabular foundation models treat each row as independent, and time-series models want regular sampling and numeric features. Neither assumption survives contact with a menstrual tracking app, which is why menstrual modeling has stayed the province of narrow, task-specific predictors of cycle length or ovulation.

This model, developed at Columbia University with Clue by BioWink and collaborators, answers that by rewriting the problem as language modeling. Each user's entire tracking history becomes one long sentence: a demographics prompt (age, categorical BMI, birth control type, year of first tracked event), then cycle blocks delimited by artificial start and end tokens, with each logged event written as a category-value token such as period:light or emotion:happy. Elapsed time between events is itself a token — D1 for a one-day gap, W7 for seven weeks, M3 for three months — so irregular spacing is carried in the vocabulary rather than dropped. A standard autoregressive transformer is then trained on these sequences with next-token prediction. The encoding extends the patient-sequence scheme of CEHR-GPT, adapted from clinical encounters to behaviorally mediated self-tracking.

The authors give the model no product name; the code release calls it the Menstrual Foundation Model. It was published in npj Women's Health in April 2026.

#Key Features

• Timelines as sentences: Cycle structure, symptom categories and elapsed time all live in a single token vocabulary, so sparse and irregularly spaced logs need no imputation or resampling before modeling.
• Prompted cycle simulation: Seeding generation with a real user's demographics and first three cycles produces synthetic continuations of their timeline, which is how the paper builds a matched synthetic cohort for comparison against real users.
• Transferable user representations: Mean-pooled last-layer embeddings, with or without PCA reduction, feed lightweight task heads and beat a symptom count-vector baseline across six forecasting tasks, with the largest margins when labeled data is scarce.
• Attack-based privacy audit: Membership inference, attribute inference and verbatim-memorization analyses are run against the generated data rather than assumed away, which matters for a data type where re-identification carries legal risk.

#Technical Details

The backbone is a standard autoregressive transformer over a learned concept-token embedding table plus trainable position embeddings, trained for five epochs with a next-token-prediction objective; pretraining runs through the CEHR-GPT framework. The study cohort is a 1,206,919-user random sample of Clue's de-identified warehouse (23,636,169 cycles, 405,461,127 tracked events), split into a disjoint 862,675-user pretraining cohort (16.9 million cycles, over 290 million symptoms) and a 344,244-user evaluation cohort from which every reported result comes. Generation uses top-k sampling with k = 100 up to a 1,024-token context. The paper states no parameter count, but Supplementary Note 3 gives 6 decoder layers, 8 attention heads and a hidden size of 768 over a 327-token vocabulary.

Synthetic and real cycles are statistically close: cycle length KL divergence 0.002 and JS 0.023, period length 0.012 and 0.054, symptom frequencies indistinguishable across all 19 categories, and the prevalence of consistently highly variable users 8.63% real versus 8.42% synthetic. Downstream numbers are modest in absolute terms — AUC 0.62 on next-cycle period flow, a 0.16 AUC gain over baseline on post-menstrual cramps, and mean absolute error of 8.90 days on next cycle length against 17.18 for the baseline. The privacy audits found membership inference at chance (ROC-AUC 0.500 ± 0.004) and a 0.82% exact-match rate for regenerating a held-out cycle, against 27.3% between naturally similar real users.

#Applications

The practical draw is a synthetic cohort that behaves like the real one without carrying individual records, letting researchers study cycle variability, symptom co-occurrence or engagement patterns, and oversample subgroups too small to analyze directly. The representation side supports personalized forecasting inside tracking apps, and gives menstrual health researchers a starting embedding for new prediction tasks instead of training from scratch on small labeled sets.

#Impact

This is the first foundation model trained at scale on self-tracked menstrual data, a domain long starved of both datasets and shared modeling infrastructure. Its limits are stated by the authors: an autoregressive objective emphasizes local transitions and does not explicitly encode cycle-level or time-to-event structure, bounding longer-horizon and phase-specific prediction; evaluation covers only the next cycle; and menstrual forecasting has essentially no task-specific benchmarks to compare against. The training population is one commercial app's users and skews young, with 87% under 33 and 66% having tracked birth control at least once. Reuse is constrained too — the analysis and evaluation code is on GitHub without a license file, the trained checkpoint has not been released, and the Clue data is available only by request under a data use agreement, so the model cannot currently be reproduced or run by third parties.