C.La.P.

Sequence-only genomic models share a structural problem: the reference genome is identical in every cell, so a model reading only sequence cannot distinguish an enhancer active in liver from the same enhancer silenced in neurons. Models such as Enformer work around this by predicting thousands of tissue-specific output tracks at once, which fixes the symptom but locks the model to the contexts in its training set. A new cell type, developmental stage or disease state remains out of reach.

C.La.P. — Chromatin LAnguage Processing — takes the other route. Developed by Panos Firbas Nisantzis, Carolina Gonçalves and Gonzalo de Polavieja at the Champalimaud Foundation in Lisbon, it puts a measured chromatin-accessibility signal into the model's input rather than asking the model to infer context from sequence. Each sample is one ATAC-seq peak — a putative cis-regulatory element — represented by its reference sequence plus the per-nucleotide ATAC-seq coverage in that biosample. Context arrives as data, so one fine-tuned model can be pointed at any cell type with an ATAC-seq experiment.

The proposition is a trade: run one cheap accessibility assay, and infer marks that would otherwise require a panel of antibody-dependent ChIP-seq experiments. The preprint was posted in February 2025.

#Key Features

• Chromatin accessibility as model input: Per-nucleotide ATAC-seq coverage, plus- and minus-strand reads kept separate, enters the input alongside the reference sequence, making every prediction specific to the assayed biosample.
• CRE-sized samples instead of long windows: Rather than feeding megabase spans and max-pooling away resolution, each sample is a single accessible region, keeping predictions at nucleotide resolution.
• Learned convolutional tokenizer: Three convolutional blocks build a 23-nucleotide overlapping window around every position, so tokens are dynamic learned motif detectors rather than a fixed k-mer vocabulary.
• Nucleotide-resolution CTCF localization: Aggregated attention peaks over CTCF position weight matrix hits, and the ATAC-seq footprint around those peaks shows the strand imbalance characteristic of a genuine binding event.
• Recovers binding events the ground truth misses: Predictions falling outside the ChIP-seq peak calls still carry that footprint signature, indicating real events the reference assay failed to capture.

#Technical Details

Input is a region of up to 1,800 nucleotides described by 28 features per position: six for the reference sequence including a mask token, two for plus- and minus-strand ATAC-seq coverage, and twenty DNA-shape features. A tokenizer of three convolutional blocks with kernel sizes 3, 9 and 13 at stride 1 gives a receptive field of 23 positions and emits one embedding per nucleotide. These pass to a bidirectional transformer encoder using ALiBi relative positional encoding and Flash Attention. Two sizes were trained: a base model with 4 layers, 12 heads and hidden size 768 (36M parameters), and a large model with 6 layers, 32 heads and hidden size 2,048 (596M parameters).

All data came from the ENCODE portal, aligned to human GRCh38; 75 ATAC-seq experiments supplied 4,688,793 pretraining samples. Pretraining is BERT-style masked-nucleotide prediction, with masked positions chosen preferentially at annotated SNPs and conserved or accelerated sites rather than uniformly at random. The base model reached validation F1 0.79 after 30 epochs; the large model reached 0.82 in five. Fine-tuning used 1.5M samples from 25 biosamples spanning 16 tissues, with three binary heads for CTCF, H3K27ac and H3K4me3 ChIP-seq. Per-token F1 for the large model was 0.79, 0.84 and 0.89 respectively; evaluated per cis-regulatory element, mean F1 across biosamples was 0.88, 0.83 and 0.85. An ablation withholding the two ATAC-seq features failed to exceed F1 0.5 in pretraining, showing the accessibility signal is load-bearing.

#Applications

The direct use is assay substitution in comparative regulatory genomics: a group profiling a tissue panel, a developmental time course or patient samples can run ATAC-seq on each and infer CTCF occupancy and active-promoter and active-enhancer marks computationally, avoiding the cost and antibody constraints of three ChIP-seq experiments per condition. The nucleotide-resolution CTCF output is useful in its own right, since ChIP-seq peaks span 200 to 400 base pairs while the bound protein occupies roughly 30.

#Impact

C.La.P. is a clean demonstration that supplying a cheap measured assay as model input buys context-specificity that no amount of sequence-only capacity provides, and that this holds across 25 biosamples rather than one held-out split. The interpretability results are unusually strong for a genomic transformer: the model rediscovers the CTCF motif without supervision and grounds its predictions in the accessibility footprint left by the bound protein. Limits are stated plainly by the authors. Accuracy drops on H3K27ac, because poised enhancers carry the mark without open chromatin and the model falls back on sequence alone; performance also degrades where ATAC-seq coverage is sparse. Training and evaluation are confined to human ENCODE data, leaving cross-species use untested, and no code or weights have been released, so the results cannot yet be independently reproduced.