scContrast

scContrast is a semi-supervised contrastive encoder for single-cell RNA-seq from the Singh lab at Brown University, presented at the AI4NA workshop at ICLR 2025. It targets a specific failure mode of large single-cell foundation models: integrating cells profiled with different sequencing chemistries. Plate-based protocols such as Smart-Seq2 and droplet-based protocols such as 10x Genomics produce systematically different expression profiles for the same cell type, and masked-modeling foundation models trained on atlas-scale corpora do not reliably erase that difference.

The approach borrows from self-supervised vision. Two augmented views of each batch of cells are generated, encoded by a small fully connected network, projected into a higher-dimensional space, and compared with a VICReg-style objective that maintains variance across cells, minimizes the difference between the two views of the same cell, and decorrelates projection features. Five augmentations do the work: injecting noise into a cell's most differentially expressed genes, permuting expression profiles among cells sharing an annotation, random dropout, random rescaling, and gene subsampling. The first two use cell-type labels, which is what makes the method semi-supervised; the remaining three simulate the technical artifacts and protocol biases the encoder is meant to ignore.

Critically, the encoder is trained once and applied to held-out tissues without retraining — the zero-shot setting that separates it from batch-correction tools such as scVI, which must be refit with explicit batch labels for every new integration.

#Key Features

• Five expression-space augmentations: Two annotation-guided augmentations preserve cell identity while three annotation-free augmentations mimic dropout and protocol bias, giving the contrastive objective biologically meaningful positive pairs.
• VICReg objective without negative pairs: Variance, invariance, and covariance terms replace explicit negative sampling, avoiding the large negative batches that contrastive methods usually require.
• No batch labels needed: Unlike supervised integration methods, scContrast never sees a batch annotation during training, yet still aligns plate- and droplet-derived cells.
• Small enough to retrain: The cell encoding module holds 5.2M parameters, 10 to 100 times smaller than transformer-based single-cell foundation models such as UCE, Geneformer, and scGPT.

#Technical Details

The encoder is three fully connected layers with batch normalization and ReLU, mapping a 20,116-gene input vector to a 128-dimensional cell encoding; a matching three-layer projection head expands this to 256 dimensions for the loss. Training used a batch size of 8,192 with AdamW, implemented in PyTorch and PyTorch Lightning. Data came from Tabula Muris, a mouse atlas of over 350,000 cells across 25 tissues profiled on both Smart-Seq2 and 10x. Twenty-one tissues were used for training; skin, liver, pancreas, and limb muscle were held out entirely. Cells with missing annotations were dropped, counts were normalized to 10,000 per cell, and expression was log-transformed.

Evaluation on the four held-out tissues used the scib-metrics suite against PCA, scVI, and UCE. On batch integration, scContrast scores 0.18 on kBET and 0.73 on PCR, against UCE's 0.02 and 0.39 and PCA's 0.00 on both; scVI, which is given batch labels, leads at 0.25 and 0.93. On bio-conservation the ordering reverses: UCE leads with KMeans ARI 0.51 and cell-type ASW 0.65, while scContrast reaches 0.43 and 0.60, close to PCA. The encoder therefore trades some cell-type separation for substantially better cross-protocol mixing.

#Applications

The natural use is integrating single-cell datasets collected under different protocols — a routine problem when combining a lab's own Smart-Seq2 data with public 10x atlases, or when merging cohorts across studies. Because the trained encoder generalizes to tissues it never saw, it can be applied to new samples without the refitting that supervised integration requires, which matters for iterative analyses where new data keeps arriving. The input is a fixed highly variable gene panel selected at training time, so applying the encoder to new data requires projecting that data onto the same gene set.

#Impact

scContrast is a workshop-scale result with a clear message: on cross-protocol integration specifically, a 5.2M-parameter contrastive encoder trained on one mouse atlas outperforms a 650M-parameter foundation model trained on 36 million cells across eight species. That reinforces independent findings that zero-shot single-cell foundation models underperform expectations on batch integration, and suggests the masked-modeling objective, rather than model capacity, is the limiting factor. The release is a research codebase rather than a product: the GitHub repository holds the model definition, SLURM training scripts, and preprocessing notebooks, but has no README, no license file, no tagged release, and no distributed checkpoint, so reproducing the results means retraining from the scripts. Evaluation is confined to one mouse atlas and four held-out tissues, and the paper has not been peer reviewed beyond workshop acceptance.