Loading…
Spatial transcriptomics foundation model aligning histology with gene expression at spot and neighborhood scale for zero-shot tissue domain calling.
No providers recorded yet. Browse all providers
A 10x Visium spot is 55 µm across, and the hematoxylin-and-eosin patch beneath it is about 28×28 pixels — a few cells' worth of texture, and far too little to tell one cortical layer from the next. Contrastive models built on the CLIP recipe pair that thumbnail against the spot's expression profile and stop there, so the objective only ever sees the spot in isolation. Whether it sits in a tumor nest, at a stromal boundary, or in cortical layer L2 rather than L1 never enters the loss, even though that placement is most of what distinguishes the regions a pathologist would draw.
ST-Align, from Xiamen University and Aginome Scientific, addresses this by treating a slide as a two-level spatial structure and aligning image to expression at both levels. Each spot is grouped with its three nearest neighbors by Euclidean distance on the slide into a niche: a larger image crop, paired with a gene vector formed by averaging the neighboring spots' profiles to approximate a locally bulk measurement. Pretraining then optimizes three contrastive terms at once — image-to-gene at the spot level, image-to-gene at the niche level, and a spot-to-niche term pulling each spot's fused embedding toward its own niche and away from others'. The third term carries slide-scale structure into the per-spot representation.
Because the result is a single pretrained encoder rather than a per-slide fit, spatial domains on a new slide come from clustering frozen embeddings, with no labels and no retraining. That is a different position from H&E-to-expression predictors such as STPath, which start from the image alone; ST-Align pretrains both modalities jointly and expects both at training time.
Pretraining draws on STimage-1K4M, filtered to human tissue sequenced on 10x Visium with at least 50 spots per slide, leaving 573 whole-slide images and 1.3 million spot-niche pairs across normal, diseased, and cancerous tissue. The niche gene encoder is a 6-layer transformer with 8 attention heads and 0.1 dropout; training uses AdamW at a learning rate of 5×10⁻⁴ under a cosine schedule with linear warmup, weight decay between 0.04 and 0.4, and synchronized batch normalization across 3 NVIDIA A800 GPUs. The total loss is a weighted sum of the two image-gene terms and the spot-niche term.
Evaluation covers six human dorsolateral prefrontal cortex slices. Clustering frozen embeddings zero-shot, ST-Align reaches an overall ARI of 0.340, against 0.264 for Scanpy, 0.262 for scFoundation, 0.256 for CLIP, 0.248 for PLIP, and 0.128 for CONCH; pathology-only backbones trail further, with Prov-GigaPath at 0.111, UNI at 0.109, and Hibou at 0.087. In a second task, an MLP fit on 80% of spots predicts nine marker genes from frozen image embeddings, where ST-Align's encoder gives an overall MSE of 0.168 versus 0.185 for Hibou and 0.184 for CLIP.
The immediate use is unsupervised annotation of spatial transcriptomics slides: cluster the frozen embeddings and read off spatial domains that track anatomical layers and tumor compartments, on a slide the model never saw. The gene-prediction results point at the more speculative use of reading expression off morphology to reduce how much of a cohort needs sequencing. For groups already running Visium, the embeddings also serve as features for retrieval, cross-section comparison, and downstream classifiers.
ST-Align is a preprint and has not been peer reviewed. Its evidence base is narrow relative to the framing: pretraining and evaluation are confined to human 10x Visium data, the clustering results come from six brain slices of one cohort, and the gene-prediction task fits a supervised head rather than testing transfer directly. A revised version of the work scopes its claims explicitly to within-platform transfer, without asserting cross-tissue or cross-technology generalization. No checkpoint has been released, the repository publishes example code rather than a runnable pipeline, and it carries no license, so the results are not currently reproducible. The design idea — that spatial neighborhood should be a contrastive target rather than a post-hoc smoothing step — is the part most likely to outlast the numbers.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.