GEM-GPT

Most generative models for drug design begin from a protein target: pick a gene, generate molecules that bind its product. That framing struggles with chronic and systemic diseases, where pathology is spread across many genes and looks different in each cell type of the affected tissue. Systems pharmacology reframes the goal as reverting an entire transcriptional state rather than inhibiting one protein, but generative chemistry built for that framing remains scarce.

GEM-GPT (Gene Expression-guided Molecular Generative Pre-trained Transformer) is a transcriptome-conditioned molecule generator built for that setting. It takes a paired diseased and healthy gene-expression profile for one specific cell type and autoregressively writes SMILES strings for compounds predicted to push that cell's state back toward health. The architecture couples a frozen single-cell foundation model — scGPT — to a trainable molecular GPT decoder through a deep fused attention mechanism applied at every decoder layer, so chemical-gene interactions are modeled cell-type-specifically throughout generation rather than injected once as a conditioning vector.

The model comes from Lei Xie's group at Northeastern University, with co-authors at Hunter College and The Graduate Center, CUNY, and is available as a bioRxiv preprint under a CC-BY-NC 4.0 license. It shares the phenotype-first premise of GEMGen, an independent transcriptome-to-molecule language model from a different group, but differs in architecture and in resolving generation down to individual cell types.

#Key Features

• Frozen single-cell encoder: scGPT's weights stay fixed during training, preserving representations learned from large-scale transcriptomic corpora while only the fusion modules and molecular decoder are optimized.
• Deep fused attention: At each decoder layer the molecular hidden state is fused separately with the encoded diseased and healthy cell states, giving the generator layer-wise access to the expression gap it must close.
• Cell-type-resolved output: The model emits distinct molecular sets for different cell types of the same tissue, quantified as a positive gap between within- and between-cell-type Jaccard overlap (Δ = 0.051 in-distribution, Δ = 0.070 out-of-distribution).
• Transfer without retraining: Trained only on compound-perturbation data, the fixed model is applied unchanged to held-out cell lines, CRISPR knockout signatures, and patient-derived brain cell types absent from training.
• Recovery of approved drugs: In an opioid use disorder (OUD) case study, generation recovered naltrexone, naloxone, and clonidine — FDA-approved drugs that were not in the training set.

#Technical Details

Training uses L1000 compound-induced expression profiles under an autoregressive negative log-likelihood objective over character-level SMILES of up to 80 tokens, with the diseased and healthy profiles encoded independently by scGPT. Two regimes are reported: an in-distribution split across three cell lines (VCAP, PC3, A375) partitioned by compound InChIKey, and an out-of-distribution split that trains on five of 20 L1000 cell lines and tests on the 15 held out. Against MolGAN, GxVAEs, GexMolGen, and MolGene-E, GEM-GPT shifts Morgan and MACCS Tanimoto and Fraggle similarity distributions toward the reference compounds in both regimes, with its largest margin out of distribution. A cross-modality test evaluates the compound-trained model on CRISPR knockout signatures for ten genes with known inhibitors (including EGFR, MTOR, HDAC1, and PIK3CA), 100 knockout and 100 control profiles each, scoring generated molecules by similarity and by Boltz-2 predicted binding probability against the corresponding protein.

One architectural point deserves stating plainly: scGPT is the only pre-existing pretrained component. The molecular decoder is trained end to end on L1000 pairs rather than pretrained independently on a broad chemical corpus, and the preprint reports no parameter counts for the decoder or the fusion modules.

#Applications

The OUD case study illustrates the intended workflow: patient-derived brain scRNA-seq defines diseased-versus-healthy contrasts in astrocytes, microglia, oligodendrocytes, and oligodendrocyte precursor cells; up to 1,000 candidates are generated per patient-cell-type pair; and Similarity Ensemble Approach target prediction links them to OUD-relevant proteins. The same pattern fits any program whose objective is a transcriptional state rather than a validated target — neuropsychiatric and other chronic diseases, repurposing searches seeded by patient expression data, and hypotheses about which cell type in a tissue to drug.

#Impact

GEM-GPT's contribution is to make phenotype-driven molecular generation cell-type-resolved, and to show that a frozen single-cell foundation model can act as a reusable conditioning encoder for chemistry — transferring from bulk compound-perturbation screens to genetic perturbations and to patient single-cell data without retraining. Recovering approved OUD medications absent from training is a useful retrospective control on that claim. The evidence is nonetheless entirely computational: similarity to known actives and predicted binding stand in for activity, no synthesis or assay data accompanies the generated compounds, the work is a preprint awaiting peer review, and no code, weights, or hosted inference have been released, which prevents independent reproduction.