Hist2Cell

An automated nucleus classifier reading an H&E section can usually manage two to four coarse cell classes. It cannot say whether the T cells in a region are CD8 effector memory or gamma-delta, or separate luminal secretory from luminal hormone-responsive epithelium. Those fine-grained transcriptional identities come from spatial transcriptomics run through a deconvolution algorithm such as Cell2location — which means paying for a spatial assay on every section you want to interpret.

Hist2Cell, from the School of Computing and Data Science and the School of Biomedical Sciences at The University of Hong Kong with the InnoHK Centre for Translational Stem Cell Biology, removes the assay from the inference path by moving it into training. Cell2location abundances are computed once over spatial transcriptomics cohorts and used as regression targets; the trained network then reads only the histology image. Because deconvolution becomes a one-time labeling step rather than a per-sample inference step, a fixed checkpoint can be pointed at slides carrying no molecular data at all. Weiqin Zhao, Zhuo Liang, Xianjie Huang, Yuanhua Huang and Lequan Yu posted it as a preprint in February 2024 and published it in Cell Genomics in 2026.

Its architectural argument is about context. Predicting from a single spot patch throws away the fact that tissue is organized; predicting a whole slide at once starves the model of training examples. Hist2Cell samples local subgraphs from a spatial graph over the slide, keeping the data abundant while preserving neighborhood structure — a different bargain from Path2Space and DeepSpot2Cell, which predict gene expression rather than cell-type abundance directly.

#Key Features

• One-stage prediction of cell types, not genes: The network regresses cell-type abundance directly instead of predicting noisy per-gene expression and deconvolving it afterward, avoiding the compounded error of the two-stage route.
• Graph attention plus transformer layers: Graph attention passes messages between a spot and its spatial neighbors, while transformer layers score any pair of spots regardless of physical distance.
• Subgraph sampling instead of whole-slide inputs: Training on randomly sampled two-hop subgraphs keeps GPU memory constant in slide size, where whole-slide transformers scale quadratically with spot count.
• Transfer to unseen cohorts: A model trained on one breast cancer cohort was applied unchanged to a second cohort from a different lab, and to hundreds of TCGA diagnostic slides scanned at different institutions.
• Super-resolved cellular maps: Sampling subgraphs at denser coordinates yields 2× to 16× finer maps than the training spot grid, either by imputing from a low-resolution map or by predicting from the image alone.

#Technical Details

Each 224 × 224 pixel patch, centered on a spot of roughly 150 µm, becomes a node in a k-nearest-neighbor spatial graph over the slide. An ImageNet-initialized ResNet-18, trained end-to-end rather than frozen, embeds each patch; a GATv2 layer with eight attention heads aggregates the two-hop neighborhood into a 256-dimensional representation; three transformer blocks model global correlations. Four heads — spot-level, local, global and fused — each emit an abundance vector, and their rectified average is trained under mean squared error. The released lung configuration predicts 80 cell types and holds 14.2M parameters.

On 11 healthy lung slides from four donors (20,770 spots), under leave-one-donor-out cross-validation, predictions correlated positively with Cell2location labels for 79 of 80 cell types, averaging a Pearson correlation of 0.31 — roughly 50% above the best image-to-expression baseline — with ciliated cells reaching 0.79. A model trained on the her2st breast cohort (13,620 spots, 8 patients) and applied without retraining to the STNet cohort (30,655 spots, 23 patients) averaged 0.29 against 0.19 for the strongest baseline, with luminal epithelial cells at 0.87. Slide-level abundance profiles fed to Cox models beat the pathology foundation model UNI on survival concordance across lung squamous, triple-negative and HER2-positive breast cancer.

#Applications

The practical target is the archive: cohorts of diagnostic slides with outcome data but no molecular profiling, common and large where spatial transcriptomics cohorts are rare and small. From those slides a group can obtain cell-type maps for colocalization analysis, tumor-region characterization and prognostic modeling, and can use integrated gradients over the predicted abundances to ask which cell populations drive a risk score.

#Impact

Hist2Cell makes the case that the useful transferable quantity from a histology image is cell-type composition rather than gene expression, and backs it with the harder evaluation: a frozen model applied to a cohort from another institution. Its limits follow from its supervision. Predictions inherit whatever bias the Cell2location labels and their single-cell reference carry, and the output vocabulary is fixed to that reference — a tissue whose cell types are absent from the atlas needs new spatial data and a retrained head, so "no retraining" holds within a tissue family rather than across biology. Predictions are spot-level estimates, not segmented cells, and the MIT-licensed repository ships lung checkpoints only.