Precious2GPT

Eight colorectal cancer cell lines in a freezer have no matched healthy colon to compare against. The differential expression analysis happens anyway, with controls borrowed from public repositories — whatever tissue, age range and platform some unrelated study collected. Precious2GPT (P2GPT) treats the missing half of the comparison as a generation problem: name a tissue, an age, a species and an omics type, and sample a plausible profile to stand in for the control that was never run.

Published in npj Aging in August 2024 by Insilico Medicine, P2GPT is a hybrid of two conditional generators rather than one network. A conditional diffusion model (CDiffusion) works on image-like renderings of an omics vector, laid out on a 2D grid by a PyDeepInsight transform so a U-Net can exploit locality between correlated genes; conditions enter as embeddings summed into the diffusion time step, with continuous age passed through a small MLP first so it is never bucketed. In parallel, a decoder-only Multi-omics Pretrained Transformer (MoPT) treats the same data as pseudo-text — each gene and its value become tokens in a bespoke vocabulary, and a generation request is literally a prompt string such as SPECIES Mouse dataset EXPRESSION TISSUE Brain MouseAGE84.

The two branches fail differently, so the model combines them per gene instead of picking a winner: MoPT scores each gene the diffusion branch produced, and Feature Weighted Linear Stacking solves for per-gene weights by linear regression within each condition subgroup. This is the second entry in Insilico's PreciousGPT lineup and a distinct architecture rather than a retrain — Precious1GPT is a supervised aging clock, and the successor Precious3GPT is a single multimodal transformer covering chemical perturbations.

#Key Features

• Two generators stacked gene by gene: weights fit per gene let whichever branch models a given gene better dominate its value.
• Conditioning on four axes at once: tissue, age, species and omics type are supplied together, and age is continuous rather than bucketed.
• Age extrapolation past the training range: trained with a hard age ceiling, the model still places generated older samples correctly beyond it in PCA space, and was used to synthesize 120- and 150-year methylation profiles for pathway analysis.
• Synthetic controls for real cases: generated controls paired with eight CRC cell lines reproduced a disease signature derived from patient data.

#Technical Details

Training used 12,453 human GTEx v8 transcriptomes (dbGaP phs000424), 12,541 mouse transcriptomes from ARCHS4 v2.2 with genes mapped to human orthologs, and 8,285 human Illumina 450k methylation samples from the CNCB repository, with β-values averaged over CpGs in each gene's TSS200 promoter window. MoPT adopts the memory-efficient MPT architecture — GPT-2-style but with positional embeddings replaced by a linear bias — instantiated from the mosaicml/mpt-7b configuration only, with no pretrained weights loaded and a custom gene/value/age/tissue/species tokenizer. Sizes follow Chinchilla scaling against corpus size, giving 4.1M, 1.7M and 1M parameters for the multi-omics, expression and methylation models respectively; sampling uses top-k 40, top-p 0.9, temperature 0.8. Most experiments run on the 978 LINCS L1000 landmark genes, with a CycleGAN extrapolating to 11,278 genes afterwards. Against CGAN, CDiffusion and MoPT alone, P2GPT gave the lowest MAE and highest R² when a CatBoost regressor trained on real data predicted age from generated data. Quality is uneven by modality: malformed samples are substantially more frequent for methylation than expression, and the rate tracks how many real samples a tissue has.

#Applications

The direct use is filling the control arm of a comparison that has none — matched profiles for case-only cohorts, cell lines, or rare tissues where real controls are unaffordable. Generated corpora also serve as privacy-safe stand-ins for restricted cohorts and as test fixtures for differential expression pipelines. In the case study, synthetic controls for eight CRC cell lines produced a meta-analysis signature correlating with a patient-derived colorectal signature at Spearman r = 0.552 on the restored gene set (0.497 on landmark genes alone), and target ranking on it recovered known CRC targets including AKT1, PTEN and CDK2.

#Impact

P2GPT's contribution is the demonstration that omics vectors can be modeled as language and as images at once, and that the two views are complementary enough to be worth combining. The limits are substantial, and the authors state most of them: the model handles tabular and image-like data only, with no route to graph-structured biology such as regulatory networks; its own complexity is called out as a barrier to replication; conditioning is confined to the tissue, species and omics vocabulary it was fit on, since the stacking weights are solved per condition subgroup; and the target-discovery step of the case study runs through PandaOmics, Insilico's commercial platform. Neither code nor weights have been released, so the model cannot be run or audited outside Insilico — a contrast with the successor, whose checkpoints are public.