SCARF

Single-cell multiome assays measure transcription and chromatin accessibility in the same cell, but the two modalities are wildly different in shape: a few thousand expressed genes against hundreds of thousands of sparse, binary accessible regions. Foundation models had been built for each modality separately — scGPT and scFoundation for scRNA-seq, peak-level models for scATAC-seq — yet none was pretrained on the paired signal itself, leaving joint analysis to integration heuristics applied after the fact.

SCARF (Single Cell ATAC-seq and RNA-seq Foundation model) is a foundation model pretrained directly on paired multiome data. It was developed at the Guangzhou Institutes of Biomedicine and Health of the Chinese Academy of Sciences with the Institute of Automation, and posted as a preprint in April 2025. The model uses a modified Mamba state-space backbone rather than a transformer, which keeps the cost of long token sequences linear and makes the hundreds of thousands of chromatin regions in an ATAC profile tractable as direct model input.

The training corpus, X-Omics, is the other half of the contribution: 2.7 million high-quality human cells reprocessed uniformly from 454 public 10x Multiome profiles, spanning 21 tissue types across developmental, homeostatic, and disease contexts.

#Key Features

• Paired-modality pretraining: RNA and ATAC are encoded by modality-specific branches and aligned by a CLIP-style contrastive objective, so the model learns both what each assay measures on its own and what the two share.
• Mamba backbone: A selective state-space architecture replaces self-attention, scaling linearly in sequence length and accommodating the long gene and peak token sequences that make multiome data expensive for transformers.
• Zero-shot cell embeddings: Cell-level <CLS> representations from either modality, or their concatenation, can be used directly for clustering and visualization without fine-tuning.
• Cross-omics translation: The fusion and generation modules predict RNA expression from ATAC-derived representations, letting one assay stand in for the other.
• Few-shot annotation: A logistic regression head on frozen SCARF embeddings, trained on 10% of a reference dataset, annotates the remaining 90% of held-out cells.

#Technical Details

Expression counts are normalized to 10,000 per cell, log1p-transformed, and rank-tokenized against corpus-wide per-gene medians following the GeneCompass strategy, over a vocabulary of 19,365 protein-coding genes. ATAC fragments are quantified against a reference peak set of 1,743,872 peaks aggregated from 254 datasets, then binarized and TF-IDF transformed. Pretraining combines masked-token reconstruction losses per modality, an inter-modality contrastive loss with a momentum queue for negative sampling, and clipping-based regularization. The model was trained for four epochs with AdamW (weight decay 0.001), a cosine schedule peaking at 5 × 10⁻⁴ after 1,000 warmup steps, batch size 32, in mixed precision across 8 NVIDIA GPUs.

On the hBrain, hPBMC, and hBMMC multiome benchmarks, SCARF produces embeddings with higher AvgBIO scores than single-modality baselines including scGPT, scFoundation, and scBasset. On zero-shot cross-modality matching it records the highest matching probability and the lowest FOSCTTM score against scGLUE, Seurat, Harmony, LIGER, uniPort, and scVI. Few-shot annotation from a 10% labeled reference reaches accuracy, macro F1, recall, and precision above 0.75, ahead of scFoundation.

#Applications

SCARF is aimed at labs generating or reanalyzing 10x Multiome data who want a single representation for both modalities. Practical uses include annotating cell types in a new multiome dataset from a small labeled reference, matching cells across unpaired RNA and ATAC experiments, imputing expression for ATAC-only cells, and producing embeddings for downstream perturbation or regulatory-network analysis. Because inference runs from released checkpoints, adopting it does not require retraining.

#Impact

SCARF is the first foundation model pretrained on paired scRNA-seq and scATAC-seq rather than adapted to multiome data after single-modality pretraining, and it is an early demonstration that state-space backbones suit single-cell data where token counts run into the hundreds of thousands. X-Omics — the uniform reprocessing of 454 public multiome profiles from raw sequencing files — is a reusable resource in its own right. Caveats are real: the preprint has not been peer reviewed, the corpus is human-only, the code is GPL-3.0 (copyleft) and notebook-centric, and the reported gains are benchmark-level rather than demonstrated in independent applications. Pretrained weights, token dictionaries, preprocessing scripts, and downstream notebooks are archived on Zenodo under CC BY 4.0, so the model runs off the shelf — but the pretraining objective is not part of the release, leaving the inference path complete and the training path not reproducible.