GRNFormer (UNC)

Single-cell RNA foundation models learn gene-gene relationships from expression counts alone. What they never see is the mechanism underneath: which transcription factors bind which enhancers, and which of those regulatory links are actually open in a given cell. That information lives in chromatin accessibility data, and it is complementary rather than redundant — a correlation between two genes in an expression matrix does not tell you which one regulates the other.

GRNFormer, from University of North Carolina at Chapel Hill with Rutgers, Barnard College, and Lawrence Livermore National Laboratory, is a structure-aware adapter that folds explicit gene regulatory networks into an existing RNA foundation model. It has two halves. A construction pipeline derives multi-scale regulatory networks from paired single-cell ATAC-seq and RNA-seq, producing both cell-type-specific and single-cell-resolution graphs. An integration module then encodes those graphs with a graph neural network and fuses the result into the backbone through a cross-attention layer that replaces the final transformer layer. The framework is backbone-agnostic: scGPT, scFoundation, and scPaLM were all wrapped and evaluated.

This entry covers the ACL Findings 2025 model from the Chen lab at UNC. It is unrelated to GRNFormer, the identically named graph-transformer from the University of Missouri that infers regulatory edges directly from an expression matrix; the two share a name but not authors, architecture, or task.

#Key Features

• Multi-scale regulatory networks: A SCENIC+-based pipeline builds cell-type-specific networks from enhancer-driven regulons, then resolves them to individual cells by thresholding AUCell activity scores with a two-component Gaussian mixture.
• Centrality-aware cross-attention: Rather than concatenating structural and expression embeddings, multi-head cross-attention reweights them, which measurably favours transcription factors — attention on TF nodes runs 2.011 times that on non-TF nodes.
• Biologically guided edge perturbation: 20% of network edges are replaced with co-expression links drawn from the same cell, so the roughly 40% of genes with no reliable regulatory annotation still receive gradient signal.
• Degree-imbalance handling: GraphSAGE's fixed-size neighbour sampling prevents high-degree transcription factors (average degree 81.3) from swamping ordinary genes (average degree 1.3).
• Works on RNA-only data: Query cells from single-modality datasets are mapped by nearest neighbour into the reference embedding space, inheriting the precomputed networks without new multi-omics profiling.

#Technical Details

Pretraining uses the Seattle Alzheimer's Disease Brain Cell Atlas, which supplies paired scRNA-seq and scATAC-seq for 113,209 cells from 28 donors across 18,984 protein-coding genes. The adapter preserves backbone dimensions — 768 hidden units for scFoundation — and pretraining objectives are unchanged from each original implementation, with scGPT and scPaLM fully pretrained on the atlas and scFoundation continued from its public checkpoint on 8 A100 GPUs. On bulk cancer drug response over CCLE and GDSC, the GRN-augmented scGPT reaches Pearson correlation of 0.906 against 0.875 for the backbone and 0.838 for DeepCDR. On single-cell drug sensitivity classification across four compounds, average AUC rises from 0.459 to 0.581 for scGPT, 0.712 to 0.743 for scFoundation, and 0.500 to 0.561 for scPaLM. Gene perturbation prediction on Adamson, Dixit, and Norman improves average delta-expression correlation from 0.381 to 0.393 for scGPT and 0.326 to 0.337 for scFoundation. Ablations show the hybrid cell-type plus cell-specific network beats either scale alone and a degree-matched random network, and that co-expression-guided edge perturbation helps where random perturbation hurts.

#Applications

The target use cases are therapeutic: predicting IC50 values for compound-cell-line pairs, classifying drug sensitivity at single-cell resolution to support personalised treatment selection, and forecasting transcriptome-wide responses to genetic perturbation for target discovery. Because the adapter is a wrapper, groups already running scGPT or scFoundation pipelines can add it without changing their pretraining recipe. The attention weights are also interpretable, offering a handle on which regulators drive a prediction.

#Impact

GRNFormer is an argument that biological prior knowledge from a second modality can be injected into a single-cell foundation model architecturally, rather than hoped for as an emergent property of scale. The gains are consistent but mostly incremental — perturbation prediction moves about one point of correlation — with the largest effect on single-cell drug classification, where scGPT gains 0.122 AUC. Two limits are stated by the authors: network quality inherits the ambiguities of motif databases and shared transcription factor binding families, and optimal network construction still requires paired multi-omics data, with reference mapping serving as the fallback for RNA-only datasets. No code, weights, or pretrained checkpoint has been released.