HEST Tissue Segmentation

HEST Tissue Segmentation is a utility model that separates tissue foreground from slide background in whole-slide histopathology images. Whole-slide images (WSIs) are gigapixel scans in which most of the canvas is empty glass, so any downstream analysis first needs a reliable mask marking where tissue actually sits. This model provides that mask, producing a binary tissue-versus-background segmentation that lets a pipeline discard blank regions and extract patches only from informative areas.

The model was released as the automated tissue-detection step of the HEST-Library, the software toolkit accompanying HEST-1k — a collection of 1,229 spatial transcriptomic profiles paired with histology images spanning 153 cohorts, 26 organs, and 367 cancer samples. HEST-1k and its tooling were introduced by the Mahmood Lab at Harvard Medical School in the NeurIPS 2024 Spotlight paper "HEST-1k: A Dataset for Spatial Transcriptomics and Histology Image Analysis" (June 2024). Within that pipeline, the segmentation model runs before patching: it isolates tissue so that image patches can be cropped around each spatial transcriptomics spot for benchmarking and multimodal training.

This is a preprocessing utility rather than a general-purpose foundation model. It performs one narrow, reproducible job — foreground masking — and is best understood as infrastructure that supports the larger HEST benchmark and the histology foundation models evaluated on it, not as a representation learner in its own right.

#Key Features

• Foreground tissue masking: Produces a binary tissue-versus-background segmentation of whole-slide images, the step that gates patch extraction in gigapixel histology pipelines.
• H&E and IHC support: Finetuned on both hematoxylin & eosin and immunohistochemistry material, so it handles the two most common stain types encountered in computational pathology.
• DeepLabV3 backbone: Uses the well-established DeepLabV3 semantic segmentation architecture with an ImageNet-pretrained ResNet50 encoder, adapted to histology through finetuning rather than trained from scratch.
• Pipeline integration: Exposed through the HEST-Library segment_tissue functionality, giving a one-call route to reproducible tissue masks inside the broader HEST workflow.
• Open weights: Model weights are published on HuggingFace under a non-commercial license, alongside the open-source HEST-Library.

#Technical Details

The model is a DeepLabV3 semantic segmentation network with an ImageNet-pretrained ResNet50 backbone, finetuned on annotated tissue regions drawn from HEST-1k and the Acrobat H&E/IHC registration dataset. DeepLabV3 uses atrous (dilated) convolutions to capture multi-scale context while preserving spatial resolution, which suits the large, texture-rich fields of a histology slide. The finetuned network outputs a tissue mask that the HEST-Library then uses to crop patches at a fixed magnification around each spatial transcriptomics spot; those patches feed the HEST-Benchmark and multimodal finetuning of histology foundation models. Training data and the model itself are distributed under the Attribution-NonCommercial-ShareAlike 4.0 International license (CC BY-NC-SA 4.0), the same license covering HEST-1k, the HEST-Library, and the HEST-Benchmark.

#Applications

The model serves researchers assembling or analyzing spatial transcriptomics and computational pathology datasets. Its immediate use is preprocessing: masking tissue so that patch extraction, quality control, and stain-aware cropping run only on relevant slide regions. Because it handles both H&E and IHC, it fits pipelines that mix stain types, and it underpins the patching that supplies HEST-Benchmark tasks and multimodal training of histology encoders. Teams building their own WSI workflows can reuse it as a drop-in tissue detector rather than annotating masks or tuning heuristic thresholds by hand.

#Impact

HEST Tissue Segmentation contributes to the reproducibility of the widely used HEST toolkit by standardizing how tissue is detected before downstream analysis, removing a common source of variation in histology pipelines. Its significance is practical rather than methodological: it packages a proven segmentation architecture into an accessible, pipeline-ready component that many groups can adopt directly. Its scope is deliberately narrow — it is a single-purpose preprocessing model, not a foundation model, and the non-commercial license limits industrial reuse. Like the rest of the HEST release, it is a research tool and has not been validated for clinical diagnostic use.