CpGPT

DNA methylation at CpG dinucleotides tracks development, tissue identity, disease, and — through epigenetic clocks — biological age. The data is awkward to model: each commercial array assays a different fixed subset of sites, from Illumina's 27k panel through EPICv2 and MSA, and the standard analysis tool is a linear predictor fit to whichever probes a cohort happens to carry. Such predictors do not transfer across platforms, tissues, or species.

CpGPT — the Cytosine-phosphate-Guanine Pretrained Transformer — reframes the problem as foundation-model pretraining. Developed by Shift Bioscience and the University of Cambridge, with co-authors including epigenetic-clock researchers Steve Horvath and Albert Higgins-Chen, it is pretrained on CpGCorpus, a purpose-built database of more than 2,000 methylation datasets covering over 150,000 samples from diverse conditions. Because each site is represented through its sequence, position, and epigenetic context rather than as a fixed column in a matrix, the model can impute and reconstruct genome-wide methylation profiles from limited input, then be fine-tuned for phenotype prediction.

Catalog neighbours differ in scope: CpG Transformer imputes missing states within single-cell bisulfite data, MethylProphet infers methylation from gene expression, and Pleiades models sequence and methylation jointly across the whole genome as a closed commercial system. CpGPT is the array- and atlas-scale option that ships openly — code, a pip-installable package, and pretrained weights are all public. It remains a bioRxiv preprint at version 4, not peer reviewed.

#Key Features

• Zero-shot reconstruction: Genome-wide profiles are reconstructed from a small number of measured sites without task-specific training, densifying sparse assays.
• Cross-platform array conversion: Beta values are imputed between Illumina panels, so cohorts built on different array generations can be analysed together.
• Context-aware site representation: Each CpG is encoded from genomic sequence, chromosomal position, and epigenetic context, letting the model generalise to unseen sites and platforms.
• Fine-tuned aging and disease heads: Checkpoints cover chronological and relative age, mortality, morbidity, cancer status, lifespan, and plasma protein levels.
• Cross-species and single-cell reach: The same backbone extends to pan-mammalian arrays and single-cell methylation, not just bulk human data.

#Technical Details

CpGPT is a transformer released in two backbone sizes, roughly 2.5M and 101M parameters, with fine-tuned variants adding a few hundred thousand parameters on top. The implementation is PyTorch and Lightning, requires Python 3.10 or newer, installs from PyPI as CpGPT, and pulls weights and per-checkpoint vocabularies from HuggingFace. Around twenty checkpoints are published: the two backbones plus fine-tuned models for age, mortality, cancer, clock proxies, pan-mammalian imputation, human RRBS atlas reconstruction, and single-cell methylation.

On aging endpoints, fine-tuned CpGPT models outperform the specialized predictors built for those tasks, including GrimAge2 and PCGrimAge for mortality and morbidity estimation. The code carries an MIT licence with one added restriction: the software and its models may not be used to enter competitions offering monetary prizes or other compensation related to aging biomarkers or longevity research — which is why GitHub classifies the repository's licence as "Other" rather than MIT. Pretraining data is distributed via S3 under the user's own AWS credentials; the weights need no such account.

#Applications

The most direct users are aging-biology and epigenetics groups: CpGPT can serve as the backbone for a new epigenetic clock instead of another penalized regression on a fixed probe set, and its fine-tuned checkpoints give off-the-shelf estimates of age, mortality risk, and disease status. Imputation and array conversion matter to anyone harmonizing cohorts assembled on different Illumina generations, a routine obstacle in meta-analyses of public methylation repositories. The pan-mammalian and single-cell checkpoints extend the same machinery to comparative-aging studies and to sparse bisulfite experiments where per-cell coverage makes imputation a prerequisite.

#Impact

CpGPT brings the pretrain-then-fine-tune pattern that reshaped protein and genome sequence modelling to DNA methylation, and pairs it with CpGCorpus, an aggregation of public methylation data at a scale the field had not previously assembled. Its practical contribution is decoupling methylation analysis from the probe set a study happened to use, which is what makes cross-platform, cross-tissue, and cross-species transfer possible. Open weights and a pip-installable package make it directly usable, in contrast to the closed epigenetic foundation models emerging alongside it. The caveats: a preprint awaiting peer review, benchmarks run by the developers, and training dominated by array-based human data.