DrugPT

Predicting how a cell's transcriptome shifts after exposure to a small molecule is a central problem in phenotypic drug discovery. The space of compound–cell-line combinations is far larger than any screening program can profile, so models that extrapolate from existing perturbation atlases have become a practical necessity for target identification, compound triage, and drug repurposing. DrugPT, developed by Linchang Zhu and Yuanhanyu Luo at the National Institute of Biological Sciences, Beijing and Tsinghua University and posted as a preprint in July 2025, is a Transformer framework that predicts a post-treatment expression profile from a cell's pre-treatment profile plus the chemical structure of the compound applied to it.

The distinguishing choice is to treat both inputs as language. Prior perturbation models largely encode compounds with graph neural networks over explicit molecular graphs and handle expression with fully connected layers or variational autoencoders — the approach taken by DeepCE, DLEPS, PRnet, and TranSiGen. DrugPT instead builds contextual embeddings with pretrained GPT-style encoders on both sides: a sequence encoder for the landmark-gene expression vector and a GPT2 encoder for tokenized chemical structure. The two embeddings are dimension-aligned, concatenated, and passed through Transformer blocks, recasting gene–drug interaction as a multimodal sequence modeling task rather than a bespoke fusion of incompatible representations.

The authors present DrugPT as a framework contribution rather than an architectural one: each encoder can be swapped, and modalities such as genetic perturbations or single-cell profiles could occupy the same sequence slots. Neighboring catalog entries take other routes — GEARS predicts responses to multi-gene CRISPR perturbations from a gene–gene graph, and MAP grounds single-cell drug response prediction in a knowledge graph.

#Key Features

• Language-model encoders for both modalities: Pre-treatment expression is embedded by a pretrained GPT-style sequence encoder capturing co-expression dependencies, and compound structure by a pretrained GPT2 encoder, avoiding explicit molecular graphs.
• Modular late fusion: The two representations are learned separately and aligned only inside the Transformer integration module, keeping each encoder replaceable.
• Zero-shot on unseen compounds: Trained on 80% of compounds and evaluated on the held-out 20%, DrugPT requires no drug-specific fine-tuning at inference.
• Generalization across cellular contexts: A separate cell-line holdout shows stable performance on lines absent from training, with predicted signatures that still cluster compounds by annotated mechanism of action in UMAP space.
• Delta-expression accuracy: The model attains the highest Pearson correlation in its benchmark set when scored on delta expression, modeling the magnitude of change rather than reproducing baseline levels.

#Technical Details

DrugPT is trained on the LINCS L1000 dataset, using the 978 landmark genes measured before and after treatment for each compound–cell pair; the pre-treatment vector is the input and the post-treatment vector the regression target. The drug encoder uses a pretrained GPT2 tokenizer and transformer layers from the HuggingFace Transformers library to produce fixed-length embeddings; the integration module is a stack of Transformer blocks with multi-head self-attention, layer normalization, and residual connections, terminating in a linear decoder. Training minimizes mean squared error against observed post-treatment expression, using Adam at a learning rate of 1e-4, batch size 64, dropout 0.1, and up to 100 epochs with early stopping, implemented in PyTorch.

Evaluation covers two independent 80/20 splits — one holding out compounds, one holding out cell lines — against DLEPS, TranSiGen, MitCP, and PRnet, scored with AUROC and F1 on binarized regulation states plus Pearson correlation on delta expression and log2 fold change. On the held-out drugs Sorafenib and Tamoxifen, predicted and observed profiles correlate at 0.86 and 0.85. The paper is candid that DrugPT does not lead on every metric; its claim is balanced performance without task-specific tuning. Parameter counts are not reported, and the code is stated as forthcoming upon publication.

#Applications

The intended use is in-silico screening of transcriptional consequences before committing assay time: ranking candidate compounds by predicted signature, exploring repurposing hypotheses against a disease expression state, and estimating responses in cell lines that are expensive or impractical to profile. Because predictions need only a baseline expression vector and a chemical structure, the model fits ahead of L1000-style experiments as a triage step. The bulk-transcriptome formulation speaks to population-average behavior, not to cell-type heterogeneity within a sample.

#Impact

DrugPT demonstrates that pretrained language-model embeddings can serve as a common representational substrate for both transcriptomes and chemical structures, a framing that simplifies adding modalities compared with graph-plus-MLP hybrids. Its practical reach is currently limited: it is a two-author preprint that has not been peer reviewed, no code or weights have been released, the preprint text carries a CC BY-NC-ND license, and evaluation is entirely in silico against LINCS ground truth. The authors note that language-model embeddings may not capture gene regulatory mechanism or drug–target interaction, and that the bulk, single-timepoint formulation omits temporal dynamics. The contribution is best read as a template for multimodal perturbation modeling rather than a finished, deployable predictor.