Colorectal H&E encoder that fuses representations learned from mIHC protein images and pseudo-spatial transcriptomics into one decodable embedding.
No providers recorded yet. Browse all providers
A pathology lab rarely has H&E, multiplex protein imaging and spatial transcriptomics measured on the same section. More often, H&E is paired with one molecular readout in one cohort and with another elsewhere, so protein-level and transcript-level views of the tissue end up in separate models. CUBE (Colorectal Universal Representation & Bridge Encoder) treats H&E as the common bridge between them. It learns one H&E representation from each pairing, then fuses the two into a single embedding that can be decoded into both kinds of molecular signal.
CUBE was developed by Zhiyan Ge and Haoyang Cai at Sichuan University and posted to bioRxiv in September 2026. It is a proof of concept for colorectal cancer. "Universal" in the name refers to combining morphology, protein and transcriptome-associated information in one representation, not coverage across tissues: training used a single colorectal dataset, and the authors state that generalization across cohorts, patients, platforms and tissues remains to be established.
CUBE never trained on measured spatial transcriptomics. Its transcriptomic targets are pseudo-ST profiles that the pretrained DeepSpot model inferred from the same H&E patches, using UNI morphology features, so the real test is what transfers to measured data.
Both branches use residual convolutional encoders with four stride-2 blocks (32–256 channels, GroupNorm, GELU). UR1 is a 256 × 32 × 32 map from 512 × 512 inputs. UR2 is a 256 × 16 × 16 map from 256 × 256 H&E, matched to a 16 × 16 × 256 pseudo-ST grid of the top 256 highly variable DeepSpot-predictable genes. The fusion module uses 256-dimensional, eight-head pre-LayerNorm attention. Training used the HEMIT colorectal dataset with its original split of 3,717 training, 630 validation and 945 test patches (1024 × 1024 pixels, registered H&E–mIHC pairs).
On the 945 HEMIT test patches, the H&E-to-mIHC branch reached a mean Pearson correlation of 0.726 (DAPI 0.711, CD3 0.536, panCK 0.931). That is close to an adapted HEMIT model (0.731) and above U-Net, ResNet and pix2pix baselines. On one 10x Genomics colorectal Visium HD section, split into contiguous spatial regions, zero-shot CUBE reached a gene-wise Pearson of 0.118 on 111 held-out patches, against 0.073 for its DeepSpot teacher, and beat the teacher on 71.3% of 247 evaluable genes. Recalibrating only the decoder on 100 training-region patches raised this to 0.150–0.167. Probes on the frozen fused representation predicted an immune–epithelial spatial enrichment score at r = 0.809 (versus 0.537 from the three concept scores alone) and a measured 17-gene KEGG ECM–receptor program at r = 0.539.
CUBE is aimed at computational pathologists and spatial biologists working with colorectal tumor tissue where only some slides carry molecular profiling. The fused embedding can be probed for immune–epithelial organization and ECM-associated programs from H&E alone. Its patch-level clusters map back onto spatially coherent regions that differ in glandular architecture, epithelial organization and cellularity. The frozen UR2 encoder can serve as a starting point for predicting measured spatial expression after a small amount of decoder calibration on a target dataset. The released code covers training and all downstream analyses; the pretrained weights sit on a Zenodo record whose files are access-restricted.
Most histology-to-molecular models learn one modality pair at a time, such as H&E to protein panels (like GigaTIME) or H&E to spatial expression. CUBE tests a different premise: separately paired datasets can be combined through a shared H&E encoder, so no single specimen needs all three measurements. It also showed a representation trained on a teacher's pseudo-labels beating that teacher on measured data, suggesting model-derived targets can act as inductive bias when measured data are reserved for validation. The evidence is narrow — one training cohort, a three-marker panel, one Visium HD section — and the work is an unreviewed preprint that sets out a design pattern for incompletely paired spatial multi-omics rather than a broadly validated tool.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.