UNI2-h

UNI2-h is the second-generation tile encoder in the UNI computational pathology line, released by the Mahmood Lab at Harvard Medical School and Brigham and Women's Hospital in January 2025. Like UNI before it, it is a frozen feature extractor rather than a diagnostic system: it turns 224x224 pixel tiles cut from whole-slide images into dense vectors that downstream models aggregate into slide-level predictions.

Both the backbone and the corpus changed between generations. UNI is a ViT-L/16 trained on more than 100 million hematoxylin and eosin (H&E) tiles from over 100,000 slides; UNI2-h is a custom ViT-H/14 with 681 million parameters and eight register tokens, trained on more than 200 million tiles sampled from over 350,000 whole-slide images in the Mass General Brigham archive. The pretraining mix widened beyond H&E to include immunohistochemistry (IHC), and the output embedding grew from 1,024 to 1,536 dimensions. In the lab's own comparison it improves on the first generation across every reported task, most sharply on 32-class pan-cancer subtyping, where TCGA Uniform Tumor accuracy rises from 0.595 to 0.675.

The release posture is unusual in one respect: UNI2-h has no paper of its own. The citable record remains the UNI Nature Medicine study, which describes a different checkpoint, and the benchmark tables live in the GitHub README rather than in peer review. Weights carry a CC-BY-NC-ND 4.0 license limited to non-commercial academic research, behind a HuggingFace access request that requires an institutional email.

#Key Features

• Mixed-stain pretraining: The corpus spans immunohistochemistry as well as H&E, so the representation is not confined to the single stain most pathology encoders see during pretraining.
• Register tokens in the backbone: Eight learnable register tokens give the transformer somewhere to park global information that would otherwise contaminate patch tokens, a fix for the high-norm artifact tokens that appear in large DINOv2-trained ViTs.
• Finer patch granularity: A 14-pixel patch at the same 224x224 input yields 256 patch tokens instead of UNI's 196, resolving smaller morphological structure per tile.
• Drop-in timm interface: A single timm.create_model call with the published construction kwargs loads the encoder and emits a 1,536-dimensional vector per tile.
• Released slide embeddings: A companion dataset ships patch features and coordinates for more than 25,000 TCGA, CPTAC, and PANDA slides as HDF5 archives, removing the tiling and inference cost from public-cohort experiments.

#Technical Details

The architecture is a custom ViT-H: 224x224 input, patch size 14, embedding dimension 1536, 24 layers, 24 attention heads, SwiGLU-packed feed-forward layers with SiLU activation at an MLP ratio of 2.66667x2, LayerScale initialized at 1e-5, and 8 register tokens, for 681 million parameters. Pretraining follows the DINOv2 recipe — DINO self-distillation with multi-crop, iBOT masked-image modeling, and KoLeo regularization on the class tokens — run in bf16 with PyTorch FSDP mixed precision on NVIDIA A100 80GB GPUs.

The lab's published comparison covers five tile-level and three slide-level tasks, and puts UNI2-h ahead of Virchow2 on seven of the eight — 0.957 on CRC-100K-Raw, 0.722 on Kather MSI, 0.711 on 30-class EBRAINS — trailing only on the internal IHC ER/PR assessment, 0.794 against 0.808. Independent evaluations place it near the top of the field without a clear lead. On the 13-task linear-probing suite drawn from eva and HEST used in the Midnight study it averages 0.776, second to Midnight-92k/392 at 0.778 and ahead of Virchow2 at 0.766, H-optimus-0 at 0.755, Prov-GigaPath at 0.752, and UNI at 0.749. On HEST, which scores spatially resolved gene expression prediction from histology across nine organs, Bioptimus puts it at 0.413 mean Pearson correlation, behind H-optimus-1 at 0.422.

#Applications

The model is used the way its predecessor is: tile the slide, extract frozen embeddings, and train a lightweight aggregator — typically attention-based multiple instance learning — for cancer subtyping, biomarker and mutation status prediction from H&E morphology, grading, survival modeling, and slide retrieval. It also serves as the vision backbone inside larger systems: GHIST+ freezes it to propagate single-cell expression from sparsely profiled tissue cores across whole slides, and HENIUM selected it over CONCH as the image-side encoder for histology-transcriptomics alignment.

#Impact

UNI2-h is the checkpoint the Mahmood Lab recommends for new work on the UNI line, and downstream papers that once reported "UNI" features increasingly mean this one. Its release was a scaling step rather than a methodological one: more tiles, more slides, a second stain, a wider backbone, and the DINOv2 objective unchanged. The repository also benchmarks a larger UNI2-g preview at ViT-g/14 scale that outscores it on most tasks but has never been released. The evidence base has real gaps: no peer-reviewed report describes this checkpoint, so its headline numbers are the developers' own; the pretraining corpus is proprietary clinical data from one US hospital system, with no datasheet and a patient population that may not represent globally diverse cohorts; pretraining code is unreleased; and the CC-BY-NC-ND 4.0 terms rule out commercial use and redistribution of derivatives.