ScPGE

Cell-type-specific gene expression is set largely by cis-regulatory elements (CREs) — enhancers, promoters, and insulators scattered across hundreds of kilobases around a gene. Sequence-to-expression models such as Enformer feed a long contiguous genomic window (196 kb) through a convolutional trunk and transformer stack. That design is expensive to train, and because it sees only DNA sequence it cannot distinguish which CREs are actually active in a given cell type.

ScPGE (scalable computational framework for Predicting Gene Expression) takes the opposite tack: instead of a continuous window, it operates on the discrete candidate CREs (cCREs) ENCODE has already annotated around each gene. For a target gene and its neighboring cCREs, ScPGE assembles three parallel 3-dimensional tensors — one-hot DNA sequence, transcription factor binding scores, and epigenomic signal tracks — then uses a convolutional network to extract per-element features and a transformer to model gene-to-cCRE and cCRE-to-cCRE relationships. Discarding the intervening non-regulatory sequence cuts the input by orders of magnitude, making training and redeployment to a new cell type tractable on one GPU.

ScPGE was developed at the Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, with collaborators at Xuzhou University of Technology, City University of Macau, and Tongji University. It was posted to bioRxiv in July 2025 and has not been peer reviewed.

#Key Features

• Discrete cCRE tensors: Sequence, TF binding, and epigenomic features from a gene and its surrounding cCREs are packed into tensors of shape 4x(m+1)x600, so 2D convolutions can process all elements in parallel without modeling the megabases between them.
• Multimodal inputs: DNase, H3K4me3, H3K27ac, and CTCF tracks combine with 600 cell-type-active TF motif scores, so the model resolves which annotated cCREs are engaged in a given cell type rather than inferring activity from sequence alone.
• Interpretable element prioritization: Gene-to-cCRE self-attention weights and in-silico perturbation of individual cCREs give two complementary rankings of which elements drive a gene's expression.
• Chromatin-loop variants: ScPGE-LP injects a HiChIP interaction matrix into the attention computation, while ScPGE-KL adds a KL-divergence loss aligning attention with a distance-decayed loop distribution; the two are strongest at distal and proximal ranges respectively.
• Adjustable scale: The number of epigenomic tracks, the number of cCREs per gene, and the depth of both stacks are hyperparameters, so the model can be sized to whatever data and compute a lab has.

#Technical Details

Human cCREs come from the SCREEN Registry v3 (1,063,878 elements), merged within 600 bp windows into 726,796 elements and padded to a uniform 600 bp; the default configuration uses the 20 nearest cCREs per gene. The feature-learning module has three stem blocks (one per modality) and four residual convolutional blocks, projecting to an (m+1)x256 embedding; the interaction module stacks four transformer blocks with T5-style relative position bias, with gene-to-gene attention masked so capacity goes to gene-cCRE relationships. A feed-forward head with softplus activation emits the expression value. Training minimizes L2-regularized MSE with AdamW at batch size 64 on a single A100, warming the learning rate from 1e-6 to 1e-3 over 5,000 steps then decaying it back over 3,000. Genes are split by chromosome: chr16 for validation, chr8 and chr9 for test.

On RNA-seq across 19 human cell types, ScPGE improves Pearson correlation by 2.5% and mean absolute error by 27% over CREaTor, and by 1.3% and 1% over EPInformer. On CAGE-seq across 10 cell types it improves PCC by 10% and MAE by 9.6% against pretrained Enformer, and by 4.2% and 2.7% against EPInformer. For enhancer-gene classification on 677 positive and 2,239 negative K562 pairs from CRISPR perturbation screens, PRAUC is comparable to EPInformer and better than Enformer and CREaTor at every distance bin.

#Applications

ScPGE is aimed at groups that need cell-type-specific regulatory predictions without the compute budget of a full sequence-to-expression model. Practical uses include shortlisting enhancer-gene pairs for CRISPRi validation, extracting TF motifs enriched in correctly predicted elements via DeepLIFT attributions, and classifying prioritized elements by SCREEN annotation type. A model trained on GM12878 transfers to K562 with only a slight loss in correlation, and human-trained models produce reasonable expression rankings on mouse cell lines.

#Impact

ScPGE reframes sequence-to-expression prediction as a set problem over annotated regulatory elements rather than a long-context sequence problem, lowering the compute floor for this class of model. The analysis of prediction categories — showing that regulatory effect decays with distance in true positives, and that near-zero DNase signal drives false negatives — is a useful diagnostic contribution in its own right. The limitations are acknowledged: accuracy drops as more cCREs are included, distal interactions remain the weak point even with chromatin loops, and cross-species transfer preserves expression rank but not absolute levels. Code and pretrained K562 and GM12878 checkpoints are on GitHub, though the repository carries no license file and the preprint is CC-BY-ND, which constrains reuse.