GeST

GeST is a generative pretrained transformer for spatial transcriptomics (ST), developed in the Department of Automation at Tsinghua University and posted as a preprint in April 2025. Transformer foundation models for transcriptomics — Geneformer, scGPT, and their successors — treat each cell as an isolated bag of gene tokens, discarding the tissue coordinates that shape how cells behave. GeST reframes pretraining around those coordinates: given a target location and the profiles and positions of the surrounding cells, predict the expression of the cell that belongs there.

This "spatially informed generation" task separates GeST from the two closest prior efforts. CellPLM is a BERT-style model that reconstructs a target cell's remaining genes from a partially observed profile, so it cannot synthesize a brand-new cell where nothing has been measured. Nicheformer pretrains on both dissociated single-cell and spatial corpora but does not use spatial information in its pretraining objective, reserving it for spatially aware fine-tuning. GeST makes neighborhood context the sole conditioning signal, which is what lets it generate unseen cells autoregressively and be perturbed in place.

After a single pretraining pass across ST datasets spanning several assay technologies and resolutions, the model transfers three ways: zero-shot niche embeddings, fine-tuned niche annotation, and in-silico spatial perturbation, in which a region's expression is edited and the model regenerates the surrounding tissue. No code repository or model weights accompany the preprint.

#Key Features

• Spatially informed generation objective: Pretraining predicts a target cell's expression from its neighbors' profiles and positions plus its own coordinate, making spatial context — not masked self-reconstruction — the learning signal.
• Spatial attention and serialization: A custom attention mask lets each prediction attend to exactly the preceding neighbor tokens plus the target position token, while cells in a cropped tissue window are ordered by distance-weighted sampling from a corner — imposing sequence order on irregular 2D data.
• Meta-cell tokenization with a hierarchical loss: Expression profiles are quantized into a discrete "meta cell vocabulary" built by PCA followed by K-means — avoiding the error accumulation that sank a continuous-regression variant — and the vocabulary is re-clustered into four nested label levels so that near-misses are still rewarded.
• In-silico spatial perturbation: Editing genes inside a region of interest and regenerating the adjacent cells reproduces measured tissue responses, extending perturbation modeling to spatial context.

#Technical Details

GeST is a transformer decoder; the default configuration uses 8 layers and 8 attention heads over a 600 µm neighbor window, with cell positions encoded by a 2D sinusoidal function. Pretraining data spans three resolutions and technologies: a single-cell-resolution MERFISH atlas of the whole mouse brain, a multi-cell-resolution Visium dataset of human primary liver cancer, and a sub-cellular-resolution Stereo-seq mouse brain section. On held-out regions, weighted-aggregation decoding reaches RMSE 1.244 on the top 200 spatially variable genes of the anterior mouse brain versus 1.289 for a Gaussian process and 1.369 for an MLP, with higher Spearman correlation across all five test sets.

Transfer results are stronger than the generative margins. On zero-shot niche clustering against Allen Mouse Brain Common Coordinate Framework labels, GeST embeddings beat the graph-based methods NicheCompass, STAGATE, GraphST, and SpaGCN on most test sets. Fine-tuned niche annotation reaches an F1 of 0.711 on anterior-brain divisions against 0.309 for scANVI and 0.106 for CellTypist, both of which classify cells without spatial context. Simulating an ischemic lesion recovered 70.11% of the 87 measured differentially expressed genes in the peri-infarct region, against 44.8% for a baseline that copies the infarct core's changes outward. Ablating the hierarchical loss, the quantization step, the neighbor information, or the ordered serialization degrades performance in every case.

#Applications

The model targets researchers working with spatially resolved tissue atlases who need niche-level rather than cell-level analysis. Zero-shot embeddings support unsupervised discovery of tissue domains in unannotated samples, and the fine-tuned classifier assigns niche labels where single-cell annotators fail because the label itself depends on neighborhood identity. The perturbation mode lets disease biologists ask how a lesion or a shifted cell population would propagate through surrounding tissue before running the experiment, and the generative head can impute expression in regions a section damaged or an assay failed to capture.

#Impact

GeST demonstrates that GPT-style generative pretraining transfers to spatial transcriptomics, a field where representation learning had been dominated by graph neural networks trained separately on each dataset. Making neighbor context the generative target rather than an auxiliary input establishes a pretraining recipe that scales with ST data volume, and ablations show performance improving with training set size. Practical uptake is limited for now: the work is an unreviewed preprint, no code or weights have been released, benchmarks cover three datasets, and the default model is modest in size — so the scaling claim rests on ablation trends rather than a large trained model.