ChromFound

ChromFound is a foundation model for single-cell chromatin accessibility (scATAC-seq), the assay that maps which stretches of the genome are open and therefore available to transcription factors. Single-cell transcriptomics has accumulated a family of foundation models — scGPT, Geneformer, scFoundation — but chromatin accessibility data resisted the same treatment for two structural reasons. Its feature space is roughly a million open chromatin regions per cell rather than twenty thousand genes, and it is far sparser, with only about 1% of loci accessible in any given cell. Worse, there is no standard vocabulary: peak sets are called per dataset, so the "features" of two experiments do not correspond, which breaks the tokenizer-sharing assumption that transcriptomic models rely on.

The model's response is a genome-aware tokenization that describes each open chromatin region by where it is rather than by an arbitrary peak identifier. A region's token embedding sums a learnable chromosome embedding, a sinusoidal positional encoding of its genomic start and end coordinates, and a linear projection of its continuous accessibility value. Because the representation is anchored to genome coordinates, peaks called by different pipelines land in a shared space and a model pretrained on one corpus can read a new dataset without re-indexing.

ChromFound was developed by the Shanghai Academy of Artificial Intelligence for Science with Fudan University, KAUST, and Zhongshan Hospital, released as a preprint in May 2025 and accepted as a poster at NeurIPS 2025.

#Key Features

• Coordinate-based tokenization: Open chromatin regions are encoded by chromosome, genomic position, and accessibility value, removing the dependence on a fixed peak vocabulary shared between training and inference.
• Hybrid long-context encoder: Mamba state space blocks handle genome-scale sequences efficiently while windowed self-attention captures local co-accessibility within neighbouring regions.
• Zero-shot cell embeddings: Cell representations transfer to clustering and annotation without task-specific retraining, which matters because labelled scATAC-seq is much rarer than labelled scRNA-seq.
• Cross-omics prediction: The learned representation supports predicting single-cell gene expression from chromatin accessibility, linking the two most common single-cell modalities.
• Regulatory inference: The model surfaces enhancer–gene links that existing computational methods miss, offering a route to interpreting noncoding disease risk variants.

#Technical Details

The encoder stacks four layers, each pairing a Mamba block with window-partitioned self-attention over 256-region windows spanning roughly 200 kb of genomic context, at an embedding dimension of 128 and a Mamba hidden dimension of 32. Each cell is represented by on the order of 440,000 accessible regions out of about 1 million genome-wide loci. Pretraining used a masked reconstruction objective with mean squared error over masked positions, running five epochs across 1.86 trillion tokens on 32 NVIDIA A100 GPUs in about 80 hours. The corpus covers 1.97 million cells from 30 tissues and 6 disease contexts, including Alzheimer's disease, Parkinson's disease, leukemia, and glioma. Across six task families the model reports a 26.2% relative gain in clustering ARI over SCALE, a 4.1% gain in Pearson correlation over scMoGNN on cross-omics prediction, and a 46.1% gain in AvgBIO over scBasset when denoising batch effects, with comparisons also drawn against CASTLE, SCALEX, Cellcano, EpiAnno, SANGO, BABEL, Harmony, and scVI.

#Applications

ChromFound is aimed at groups working with scATAC-seq who currently retrain a bespoke model for each dataset: annotating cell types in chromatin data where marker-gene heuristics do not apply, integrating experiments across batches and tissues, imputing transcriptomes for samples profiled only for accessibility, and prioritising noncoding GWAS variants by tying enhancers to their target genes. It complements Atacformer, which tackles the same modality with a transformer over cis-regulatory elements.

#Impact

By making a coordinate-based vocabulary work at genome scale, ChromFound removes the peak-set mismatch that has kept chromatin accessibility from benefiting from transfer learning the way transcriptomics has. Code and pretrained weights are both public under Apache-2.0, the weights hosted on Hugging Face and mirrored on Google Drive, and the NeurIPS 2025 poster acceptance provides peer review of the method, though no journal version has appeared. Adoption is early: the repository is new and lightly starred, and reported benchmarks are computational, with the enhancer–gene links inferred rather than experimentally validated.