GPCT

Two immunology labs can profile the same mouse and produce data no single model can read. One panel measures B220, CD3, CD4, CD44, IgD, IgM, KLRG1 and NK1-1; the next swaps KLRG1 for CD25. A transformer expecting a fixed feature vector sees two different input spaces, so the standard remedy is one model per panel — which strands each model on however many samples that panel happens to cover. Decades of flow cytometry data have accumulated this way: abundant in aggregate, but fragmented into pockets too small to train on.

GPCT, the Generalised Pretrained Cytometry Transformer from the Australian National University, attacks that fragmentation with the Universal Cellular Embedding of Marker expression (UCEM). Each cell is written as a sparse expression vector over a fixed 26-marker vocabulary and concatenated with a one-hot indicator of which markers the assay actually measured. That separates "not measured" from "measured and absent", the distinction a zero-padded vector destroys, and places every panel in one fixed-size representation space.

The embedded cells then pass through an encoder-decoder transformer, pretrained by self-supervised masked prediction on unlabelled samples and frozen thereafter, with only a small task-specific decoder fitted per downstream question. It is one of two recent panel-agnostic cytometry foundation models: EventHorizon takes a self-distillation route on clinical human specimens, while GPCT works by masked prediction on mouse immunophenotyping cohorts and treats per-cell interpretability as a design goal.

#Key Features

• Universal Cellular Embedding of Marker expression: A learnable embedding that turns any antibody panel into a fixed-size per-cell vector by pairing sparse expression values with an explicit marker-availability mask.
• Cytometry-specific pretraining targets: Masked prediction recovers three per-cell quantities — raw marker expression, the cell's percentile rank within the sample's distribution for that marker, and a log-scaled kernel density estimate of the same. The two sample-relative targets are less sensitive to the signal shifts that batch effects and fluorophore changes introduce.
• Dual masking: Uniform random masking pushes the encoder to infer values from other cells; marker-wise masking, which hides a whole channel, pushes it to learn marker co-expression.
• Frozen encoder, lightweight task head: Downstream training updates only a cross-attention decoder and its prediction head, so a new phenotype question costs a small fit rather than a full retraining.
• Per-cell attribution: The decoder pools cells by cross-attention against a learnable prediction token, and those weights read directly as per-cell importance.

#Technical Details

GPCT stacks four encoder and four decoder layers with four attention heads and a hidden size of 128, in Pre-LN form and without positional encoding, since a cytometry sample is an unordered set of cells. Pretraining and downstream training subsample each specimen to 5,000 and 7,000 cells respectively. Two mouse cohorts anchor the evaluation: 14,014 samples from an ENU mutagenesis programme at the Australian Phenomics Facility, spanning 16 marker panels drawn from 21 markers with only B220 and CD3 measured throughout; and roughly 7,000 Knockout Mouse Project T-cell panel samples from FlowRepository. On biological sex classification with batch-grouped 7-fold cross-validation, the pretrained-encoder model reached 0.870 accuracy and 0.938 AUC, against 0.838/0.906 for a decoder-only ablation and 0.802/0.876 for the same architecture without pretraining. Holding out an entire marker panel cost under eight points of accuracy relative to the full-data baseline. On a five-class gene-knockout task built from just 72 samples, an encoder pretrained on 6,904 unrelated KOMP mice reached 0.919 macro-average AUC and 0.736 accuracy under leave-one-batch-out cross-validation, and stayed above chance from a single labelled sample per class, where a task-specific pretrained encoder did not.

#Applications

Immunophenotyping cohorts are the immediate fit: screening programmes that push thousands of animals through drifting panels, where the question is a sample-level label — genotype, sex, treatment arm — rather than a per-cell one. The frozen-encoder design suits groups holding large unlabelled FCS archives alongside a handful of annotated samples, since the expensive stage never sees a label. The attention maps also give an immunologist something checkable: the authors traced sex classification to IgM+ IgD+ B cells and an NK1-1+ KLRG1+ population, a data-driven route to refining manual gating.

#Impact

GPCT establishes that a panel-agnostic representation learned without labels transfers across cytometry datasets, and that this transfer is what rescues the small-cohort tasks dominating practical immunophenotyping. The scope is bounded: everything is mouse, the marker vocabulary is capped at 26, and the pretraining corpora number tens of thousands of samples rather than the millions language and vision foundation models draw on. The title says "towards" a cytometry foundation model for that reason, and names human clinical data — where panel diversity and sample scarcity are sharpest — as the next step. Batch effects are absorbed implicitly by the pretraining targets rather than explicitly corrected. The work remains a preprint awaiting peer review, with source code released under GPL-3.0 and both the ENU dataset and the trained weights deposited on Zenodo.