PerturbPFN

Predicting how a cell's transcriptome shifts in response to a drug it has never encountered is one of the harder extrapolation problems in single-cell biology: the compound's protein targets are often unknown, the response is high-dimensional, and measured perturbations cover a vanishing fraction of small-molecule space. Most learned approaches regress expression responses from large perturbation atlases — the strategy behind CPA, PerturbDiff, and STATE. PerturbPFN takes the opposite route: it sees no real measurements during training at all.

Built by researchers at the University of Cambridge with Prior Labs, it is a prior-fitted network — an amortized Bayesian predictor meta-trained on episodes sampled from a generative prior, the lineage of tabular PFNs and, in biology, cfRNA-ICL. Biologically motivated simulators sample a latent regulatory graph over genes, a sparse set of intervention targets, intervention strengths, and the expression readouts that follow; having learned to invert that process, the network conditions on observed dose–response regimes supplied in context and predicts held-out ones with no gradient updates at test time.

What separates PerturbPFN from direct response regressors is that it never predicts the expression vector head-on. It infers the latents first — system graph, atomic intervention targets, intervention strengths — then propagates them through a structural causal model (SCM) decoder. That indirection makes the intermediates legible: the same forward pass that predicts a response also nominates which genes the compound acts on and how hard it pushes them.

#Key Features

• Synthetic-only pretraining: The model is meta-trained entirely on prior-predictive episodes from graph and expression simulators, with no real perturbation measurements in the training set.
• Structured in-context learning: New cell line–drug protocols are handled by conditioning on context regimes at inference, without fine-tuning or test-time gradient steps.
• Interpretable intermediates: Target identity, intervention strength, and regulatory structure are explicit model outputs rather than post-hoc attributions.
• Low inference cost: In-context prediction runs in roughly a second per evaluation protocol on the reported benchmarks.

#Technical Details

PerturbPFN is a 14.98M-parameter transformer stack: a shared encoder (192 hidden dimensions, 6 attention heads, 5 blocks), a structure encoder (256 dimensions, 8 heads, 4 blocks), and a two-layer SCM decoder with 64 hidden dimensions. It handles up to 60 graph nodes plus an 11-dimensional metadata vector, and each training episode comprises 20 context regimes and 4 query regimes over 40–60 simulated genes, trained for 3,500 steps at batch size 8. On in-prior episodes it recovers its latents well: graph AUROC 0.8133, target AUROC 0.9330 (F1 0.7649), intervention-strength ratio 0.9386.

Real-data evaluation uses Sci-Plex — 12 cell line–drug combinations across A549, K562, and MCF7 with the HDAC inhibitors belinostat, dacinostat, givinostat, and quisinostat, under a highest-dosage holdout. PerturbPFN attains the best normalized Wasserstein-2 score of the compared methods (0.9183) and a Pearson-correlation score of 0.8813, ranking second overall behind a random forest X-learner while needing only 1.11 ± 0.34 seconds of in-context inference per protocol; other baselines span generative intervention models, conditional optimal transport, an MLP, and naive dose- and control-matching. On the GeneRNIB 300BCG network-inference benchmark (38 donors) it records the top GS-F1 (0.6410) among methods including GRNBoost2, SCENIC, PORTIA, and PPCOR.

#Applications

The model suits early screening questions where a compound's mechanism is unresolved: given a few measured doses in one or two cell lines, it extrapolates to unmeasured doses while nominating candidate targets and a regulatory structure to test. Because a prediction costs about a second, it can be swept across many cell line–drug–dose protocols to triage which merit wet-lab follow-up. Its scope is small gene panels rather than whole transcriptomes, which fits pathway-focused analysis and mechanism-of-action hypothesis generation alongside atlas-scale predictors such as MAP.

#Impact

PerturbPFN probes how far a purely synthetic prior can carry drug-perturbation modelling, and the answer it reports is partial: competitive prediction at low inference cost with interpretable latents, framed as a complementary trade-off rather than a win over specialized baselines. The authors are explicit that the meaning of the latent variables is set by the synthetic prior, so recovering them in-prior does not establish that they correspond to identifiable biological mechanisms. Other constraints are real — point estimates without uncertainty quantification, an assumption of sparse acyclic systems, small gene panels, and no design for static network annotation. It is a workshop paper, and no code or weights have been released. The contribution is directional: amortized, prior-fitted inference is a viable third path alongside atlas-trained regressors, and the bottleneck sits in the realism of the simulators.