PathVQ

A pathology whole-slide image is a gigapixel scan, so essentially every computational pathology pipeline breaks it into tiles, runs a pretrained vision transformer over each tile, and pools the resulting tile vectors with a multiple-instance learning head. The compression happens at one specific point: each 224×224 tile is reduced to the single [CLS] token of its ViT, throwing away the 196 spatial patch tokens that encode where things are inside the tile. Keeping those tokens helps downstream tasks, but doing so naively costs up to 200× more storage and compute — a trade-off between efficiency and representational richness that has capped how far tile-level foundation models scale into slide-level work.

PathVQ, from Westlake University and Zhejiang University, attacks that trade-off with vector quantization. A distillation stage learns to compress each patch token from 1024 dimensions to a 16-dimensional discrete code drawn from a shared codebook, then reconstruct the original token with a transformer decoder — a 64× compression that keeps reconstruction fidelity high enough for downstream use. The method was presented as a poster at NeurIPS 2025.

The second half of the contribution reuses those codes as supervision. Because quantization turns patch features into discrete indices, the quantizer functions like a tokenizer, and predicting indices becomes a BERT-style pretext task — which gives slide-level pretraining an objective that does not depend on scarce image-text pairs.

#Key Features

• Vector-quantized patch distillation: An MLP encoder maps frozen tile-encoder patch features into a codebook of 8,192 entries at embedding dimension 16, with a transformer decoder trained to reconstruct the original features.
• Multi-scale quantization: Quantizing across a resolution list of 1×1, 2×2, 4×4, 7×7, and 14×14 captures tile-level and patch-level structure in one pass, improving reconstruction and supplying targets at several granularities.
• Slide-level self-supervised pretraining: Quantized indices act as supervision for a slide-level pretext task, so a slide encoder can be pretrained on unlabeled scans without paired reports.
• Progressive convolutional aggregation: Four stride-2 convolutions widen channels from 128 to 256 to 512 to 1024, folding a region's quantized grid into a spatially enriched slide-level embedding.
• Encoder-agnostic drop-in: The tile-level backbone stays frozen, and the compressed features feed existing heads unchanged, including attention-based MIL and transformer aggregators.

#Technical Details

Quantization is pretrained on 1M randomly cropped 224×224 tiles from the TCGA diagnostic slide collection with UNI's ViT-Large backbone frozen, using four RTX-3090 GPUs for 50 epochs in roughly 22 hours. Slide-level pretraining then runs over about 250,000 regions of 3,584×3,584 pixels cropped from the same collection; storing a region as quantized indices takes only about 65MB, and re-embedding yields a tensor per region that the convolutional module reduces to a single 1024-dimensional vector before an ABMIL or six-layer WSI-Transformer head.

On BRACS three-class breast tumor classification, adding PathVQ to UNI features under ABMIL raises macro-F1 from 0.692 to 0.730 and AUC from 0.875 to 0.902, reaching 0.747 and 0.906 with slide-level pretraining. On TCGA LGG-GBM R132 mutation prediction the same progression runs 0.685 to 0.723 to 0.752 in F1. For comparison, UNI-2 with ABMIL reaches 0.698 F1 on BRACS, Prov-GigaPath 0.677, and TITAN 0.696. Disease-specific survival prediction across five TCGA cohorts improves the ABMIL c-index from 0.633 to 0.655 on BRCA, 0.612 to 0.649 on CRC, 0.540 to 0.608 on BLCA, 0.671 to 0.721 on UCEC, and 0.691 to 0.760 on KIRC, with CHIEF among the slide-level baselines.

#Applications

The method suits groups already running a tile-level pathology foundation model over large slide archives: subtyping and grading tasks where spatial arrangement inside a tile matters, mutation prediction from H&E morphology, and prognostic modeling from resection slides. Because the quantizer sits between a frozen encoder and an unchanged aggregation head, it can be inserted into an existing feature-extraction pipeline, and the discrete-index representation shrinks the feature store enough that slide-level pretraining becomes feasible on modest hardware.

#Impact

PathVQ's central argument is diagnostic rather than incremental: it attributes the plateau in slide-level performance to the [CLS] bottleneck rather than to model or data scale, and supports that with a direct comparison — adding 1M tiles of quantization training to UNI buys roughly a 3-point F1 gain on BRACS, while UNI-2 buys roughly 1 point despite far more pretraining data and a model two to six times larger. The authors note two limitations: the quantization stage requires its own additional training data, and quantization still loses some information. Evaluation is retrospective across public cohorts, with no prospective or clinical assessment. The full code implementation is announced but has not been released, and no weights are available.