GRNFormer

Reconstructing gene regulatory networks (GRNs) — which transcription factors control which target genes — is one of the oldest open problems in computational biology, and single-cell RNA-seq made it both more tractable and more difficult. Expression matrices are sparse, high-dimensional, and noisy, and nearly every established inference method is fit separately on each dataset it is applied to, often requiring cell-type labels or a prior regulatory scaffold. That per-dataset refitting is what has kept GRN inference from behaving like the rest of modern computational biology, where a pretrained model is downloaded and run.

GRNFormer, developed by Akshata Hegde and Jianlin Cheng at the University of Missouri, is a graph transformer trained once and then applied directly to new expression matrices. It takes only a gene expression matrix and a list of transcription factors, requires no cell-type annotations and no prior regulatory information, and emits a scored TF-to-gene edge list. The released checkpoint transfers across species, cell types, and even assay modality: a model trained exclusively on mammalian single-cell data was applied unchanged to bacterial and yeast bulk RNA-seq.

This makes GRNFormer a different kind of tool from single-cell foundation models such as scPRINT or Geneformer, which learn general cell and gene representations from which regulatory structure can be read out. GRNFormer is trained end-to-end on the network reconstruction objective itself. It was first posted as a preprint in January 2025 and has not yet completed peer review.

#Key Features

• Zero-shot GRN inference: A single released checkpoint is applied directly to a new expression matrix, with no retraining or per-dataset fitting.
• TF-Walker subgraph sampling: A transcription-factor-anchored random walk extracts fixed-size local neighborhoods from the gene co-expression graph, so inference cost does not scale with the size of the full network.
• Dataset-invariant expression encoding: The Gene-Transcoder module maps variable-length expression profiles — any number of cells, any dynamic range — into fixed-length gene embeddings, letting one checkpoint span platforms.
• Cross-modality transfer: The same model handles single-cell and bulk RNA-seq, and generalizes from mammalian training data to prokaryotic and yeast test data.
• No prior knowledge required: Neither cell-type labels, clustering results, nor an existing regulatory network are needed at inference time.

#Technical Details

The architecture has three stages. Gene-Transcoder is a transformer encoder that takes a subgraph expression matrix of 100 genes by a variable number of cells, plus an appended TF-identity flag, passes it through a 1D convolutional layer, and produces 64-dimensional node embeddings invariant to dataset-specific scale. GraViTAE, a variational graph transformer autoencoder, consumes those embeddings with pairwise co-expression values as edge features; its encoder stacks four Transformer Convolution blocks with 4 attention heads each and projects into a 16-dimensional latent space, and its decoder applies three further blocks plus a lightweight MLP to yield updated node embeddings and edge attentions that drive the final edge-probability head. Subgraph-level predictions are aggregated into a global network.

Training used BEELINE-curated scRNA-seq from five cell types (hESC, hHep, mDC, mHSC-E, mHSC-GM), each paired with three reference networks — cell-type-specific ChIP-seq, non-specific ChIP-seq, and STRING — on two NVIDIA A10 GPUs. Evaluation was deliberately harder for GRNFormer than for its baselines: it was held out on two unseen cell types (mESC, mHSC-L), while the nine comparison methods (CNNC, GNE, GNNLink, STGRNS, scGREAT, LEAP, PIDC, PPCOR, SINCERITIES) were trained within each test dataset per their original protocols. GRNFormer reported sampled AUROC and AUPRC of 0.90–0.98 and sampled F1 of 0.87–0.98 across BEELINE subsets, and on DREAM5 bulk data reached sampled AUROC 0.979 (E. coli) and 0.977 (S. cerevisiae). Inference runtime scaled smoothly from roughly 500 to 5,900 input genes.

#Applications

The practical case is a lab with an expression matrix and no ground-truth network: run inference and get a ranked regulatory edge list. The authors demonstrate recovery of pluripotency circuitry in human embryonic stem cells and, in a blind run on the 10x Genomics PBMC 3k dataset with no cell-type labels, immune lineage modules whose pathway enrichment matched known PBMC biology. Because the checkpoint transfers to bulk RNA-seq and to microbial data, it also serves comparative and microbial genomics settings where curated regulatory references are thin or absent.

#Impact

GRNFormer's contribution is less a benchmark number than a change in workflow: it packages GRN inference as a downloadable checkpoint rather than a fitting procedure. The code is MIT-licensed, the trained weights ship in the repository, and the manuscript version is archived on Zenodo. The limitations are stated plainly by the authors — accuracy degrades on datasets with extreme sparsity or weak transcription factor signal, and the reliance on local co-expression neighborhoods means distal and chromatin-level regulation such as enhancer-promoter looping is invisible to the model. There is no hosted API or package release; use requires cloning the repository and running the Python scripts locally.