OmniPert

Perturbation screens — CRISPR knockouts, activations, and small-molecule treatments read out by single-cell RNA sequencing — have become the workhorse of target discovery in oncology, but the space they aim to cover cannot be tested exhaustively: every additional cell type, cell state, perturbation target, and modality multiplies the experiments required. OmniPert simulates those experiments in silico, predicting an individual cell's post-perturbation expression profile from its pre-perturbation transcriptome and a description of the intervention.

What separates OmniPert from earlier single-cell perturbation models is that it handles genetic and chemical interventions inside a single architecture, including cases where both are applied to the same cell. CPA and scGPT established that deep learning can predict transcriptional responses to perturbation, but each addresses one modality at a time. OmniPert fuses embeddings of gene expression, gene identity, perturbation target, and perturbation type on a biological input head, embeds small molecules on a separate chemical head, and couples the two through cross-attention.

Farzan Taj and Lincoln Stein developed the model at the University of Toronto, the Ontario Institute for Cancer Research, and the Vector Institute, posting it to bioRxiv in July 2025. It extends their earlier bulk-omics drug response predictor, MMDRP, to single-cell resolution and to genetic perturbations.

#Key Features

• Six perturbation modalities in one model: CRISPRa activation, CRISPRi repression, CRISPR-Cas9 knockout, CRISPR-Cas13 knockdown, ORF overexpression, and chemical treatment, plus multi-gene, multi-drug, and mixed gene-drug combinations.
• Cross-attention between drugs and genes: Small molecules are encoded with the Uni-Mol+ molecular property predictor, then joined to the transcriptomic representation by a cross-attention layer that pairs each compound with every expressed gene in the cell.
• Interpretable interaction networks: Attention rollout converts the self- and cross-attention matrices into weighted gene-gene and gene-drug graphs, recovering known drug synergies and pathway members such as the ERK signaling nodes engaged by combined MAPK7 knockdown and trametinib.
• Counterfactual perturbation search: Given a target transcriptional state, OmniPert ranks the candidate interventions that would produce it. The true perturbation fell at an average normalized rank of 0.29, against 0.50 for a random baseline.
• Out-of-distribution validation design: The 90:10 split is taken over unique perturbation, cell line, and dataset triplets, so held-out samples demand generalization to new contexts rather than recall of seen conditions.

#Technical Details

The main transformer module has three layers with eight attention heads each, the cross-attention module is a single eight-head layer, and the latent dimensionality throughout is 256. Training minimizes a mean squared error term plus a tanh-based directional loss, in PyTorch and PyTorch Lightning, with no self-supervised pretraining stage. Training data come from 20 single-cell perturbation studies harmonized from the scPerturb database, an aggregate of over 6.25 million cells: roughly 4 million genetically perturbed, 1.1 million chemically perturbed, 617,000 with simultaneous genetic and chemical perturbation, and 490,000 controls. Filtering to cells with 100 to 5,000 expressed genes left about 5.7 million — 5.2 million perturbed cells for training and 555,000 for validation. The chemical library spans 211 compounds.

On held-out data the model reaches a coefficient of determination near 0.75 across five perturbation types and roughly 0.88 for ORF overexpression, whose training data come from a single experimental source. Median per-sample Pearson correlations were 0.740 for multi-gene combinations (n = 14,892), 0.801 for multi-drug (n = 11,321), 0.780 for mixed gene-drug samples (n = 99,364), and 0.764 for single perturbations; single-drug samples were weakest at 0.492, across a validation set spanning over 200 cell lines. Genetic and chemical perturbations aimed at the same target gave predicted profiles 66.6% to 73.6% closer in cosine distance than random gene-drug pairings.

#Applications

OmniPert is aimed at precision oncology workflows where the experiment of interest is too large to run: prioritizing which CRISPR or compound perturbations to test in a screen, nominating cell-type- and cell-state-specific drug targets, inferring a compound's biologically relevant target from its attention-derived gene network, and finding less toxic compounds that phenocopy an existing drug. The counterfactual mode inverts the question, asking which intervention drives a cell toward a desired state — differentiation of a cancer stem-like population, for instance.

#Impact

OmniPert is, on the authors' account, the first transformer to embed genetic and small-molecule interventions jointly at single-cell resolution and model their combined effects, and it shows that training this way encodes concordance between a drug and the genetic perturbation of its target. Several constraints bound its current use. The work is a preprint that has not been peer reviewed, all benchmarks are in silico against held-out screen data rather than prospective experiments, and the code availability statement points to a GitHub repository that is not publicly accessible, with no released weights. The architecture assumes a one-to-one correspondence between pre- and post-perturbation gene sets, imputing zeros for genes silent before perturbation, so it cannot predict genes newly activated by an intervention.