H&Enium

Spatial transcriptomics platforms such as 10x Genomics Xenium can assign gene expression to individual segmented cells on a hematoxylin and eosin (H&E) slide, but they remain expensive and low-throughput relative to routine H&E staining. H&Enium addresses that asymmetry: it injects transcriptomic context into image-derived cell embeddings during training, so that at inference time cell identity and gene expression can be read out from morphology alone.

Developed by the Boeva lab at ETH Zurich with a co-author at the Dana-Farber Cancer Institute, H&Enium is an alignment framework rather than a model trained from scratch. Two frozen foundation models supply the inputs — a pathology model that embeds a 224x224 pixel patch centered on each nucleus, and a transcriptomics model that embeds the cell's gene expression vector. Only two lightweight multilayer perceptron projection heads are trained, pulling matched image and expression embeddings together in a 128-dimensional shared space with a CLIP-style contrastive loss. The authors benchmarked UNI (using the UNI2 checkpoint) against CONCH on the image side and CellPLM against scGPT on the expression side, selecting UNI2 and CellPLM as the backbones.

The methodological contribution is a soft alignment target, BLEEP_input. Earlier CLIP-derived methods for histology and spatial transcriptomics operated at spot-level resolution and used either hard one-hot targets or soft targets computed from the projected embeddings. H&Enium instead computes intra-modality similarity from the pre-projection foundation model embeddings, which are more stable early in training. The work was presented at the ICML 2025 Workshop on Multi-modal Foundation Models and Large Language Models for Life Sciences and posted as a bioRxiv preprint; it has not been peer reviewed, and no code or weights have been released.

#Key Features

• Single-cell resolution alignment: Operates on individual segmented cells rather than the multi-cell spots used by earlier image–expression contrastive methods, the scale at which cell-type identity and microenvironment interactions are defined.
• Frozen backbones, trained heads: Both foundation models stay frozen and only two one-layer projection heads are optimized, making alignment cheap and swappable across upstream models.
• BLEEP_input soft target: Builds the target similarity matrix from pre-projection foundation model embeddings, which outperformed both the one-hot CLIP target and the original BLEEP target on out-of-sample data.
• H&E-only inference: Once aligned, the image projection head requires no transcriptomic input, so transcriptomically informed embeddings can be computed for archival H&E slides.
• Out-of-sample validation: Models trained on Xenium slides were evaluated on the independent PanNuke dataset, whose labels come from pathologist review of images rather than from expression.

#Technical Details

Training used three publicly available 10x Genomics Xenium tumor slides — pancreas (190,965 cells, 474-gene panel), breast invasive ductal carcinoma (574,527 cells) and breast invasive lobular carcinoma (365,604 cells, both 380-gene panels) — leaving roughly 858,000 cells after quality filtering. Cells were labeled with four PanNuke classes (connective, inflammatory, neoplastic, epithelial) by expert annotation of the expression data alone, and evaluation used five-fold spatial cross-validation. Each projection head is a single layer producing 128-dimensional output with GELU activation, layer normalization and dropout of 0.3, trained with AdamW for up to 20 epochs at batch size 64.

Unaligned UNI2 embeddings gave cell-type F1 scores of 0.52 (breast ILC) to 0.74 (breast IDC), well above a handcrafted morphological baseline (0.39–0.48) but far below the 0.86–0.94 achieved from expression embeddings. Alignment with BLEEP_input improved in-distribution F1 modestly (about 7% relative on breast ILC, under 1% on breast IDC). The larger gains appear out of sample: on PanNuke, F1 rose from 0.2241 to 0.2641 (pancreas), 0.4977 to 0.5804 (breast IDC) and 0.3558 to 0.4759 (breast ILC), improvements exceeding 16% relative. Gene expression prediction from image embeddings, scored on the 50 most variable genes with ridge regression, improved Pearson correlation from 0.3612 to 0.4012 (pancreas), 0.4313 to 0.4755 (breast IDC) and 0.3337 to 0.3698 (breast ILC), with matching gains in relative variance distance.

#Applications

The intended use is enriching standard H&E analysis pipelines in oncology research. Because inference needs only an image patch, aligned embeddings can be computed retrospectively for large archival whole-slide image cohorts that have no paired spatial transcriptomics, supporting single-cell type mapping and spatial characterization of the tumor microenvironment at a fraction of the cost of running Xenium. The frozen-backbone design also makes the framework a template for combining a group's preferred pathology and transcriptomics models without retraining either.

#Impact

H&Enium is, by the authors' account, the first alignment of H&E imaging and transcriptomics at single-cell rather than spot resolution, and it demonstrates that patch-level pathology foundation models transfer usefully to nucleus-centered crops. The scope is deliberately narrow: three slides from two tumor types, four coarse cell classes, and absolute out-of-sample performance that remains low (best PanNuke F1 of 0.58). It is a workshop-scale proof of concept rather than a deployed tool, and with no released code or checkpoints, reproduction requires reimplementation from the paper.