CAPT

Calcium imaging routinely records the activity of tens of thousands of individual neurons at once, and the resulting archives are large enough to support foundation-model-style pretraining. Most models in this space, however, are fit per dataset: trained on one collection of recordings and evaluated on held-out sessions from that same collection. That leaves the central question untested — whether the dynamics such a model learns are general, or an artifact of one lab's indicator, frame rate, preparation, and task structure. CAPT (Continuous Autoregressive Population Transformer) was built to answer it.

Developed by Xinhong Xu, Yimeng Zhang, and Yuanlong Zhang at Tsinghua University, CAPT treats calcium traces as continuous signals rather than discretizing them. Each single-neuron trace is partitioned into short temporal patches, projected into the model space together with learned neuron and session embeddings, and predicted autoregressively under a mean-squared-error objective. Keeping the representation continuous preserves the graded amplitude information that a vector-quantized tokenizer discards — a different bet from tokenizer-based brain-signal foundation models such as LaBraM, and from masked-reconstruction approaches like BrainMAE.

The protocol is what gives the result its force. CAPT is pretrained once, on mouse data only, and in every transfer experiment the pretrained backbone is frozen: only adaptation modules — the neuron and session embeddings and the task head — are updated. Under that constraint it is evaluated on eight independent datasets from other labs, spanning mouse, larval zebrafish, and C. elegans.

#Key Features

• Continuous patch tokenization: Traces are modeled as continuous values in short temporal patches, avoiding the discretization step used by most brain-signal foundation models and retaining amplitude structure.
• Dual-axis attention: The backbone pairs causal temporal attention, which models each neuron's own history, with neuron-axis attention, which models population-level interactions at each patch position.
• Frozen-backbone transfer: Downstream adaptation touches only embeddings and a decoding head, so gains on new datasets are attributable to the pretrained representation rather than to refitting the model.
• Cross-species evaluation: One mouse-pretrained checkpoint is tested on mouse, larval zebrafish, and C. elegans recordings collected under different imaging conditions and paradigms.
• Shared functional embedding space: Neuron embeddings from separate datasets occupy a common space that carries anatomical cell-identity information, not just dataset-specific structure.
• Multi-task backbone: One checkpoint supports both population forecasting and behavior decoding.

#Technical Details

The Dual-axis Transformer (DAT) backbone uses 6 layers, a model dimension of 512, 8 attention heads, a feed-forward dimension of 2,048, and a patch length of 8 frames. Pretraining draws on a single large-scale mouse dataset comprising 286 sessions from 8 mice — 273,770 neurons recorded across 6 brain regions and 2 cortical layers. Transfer evaluation adds three mouse datasets (52 sessions/212,464 neurons; 3/988; 3/9,619), three larval zebrafish datasets (45/140,311; 29/4,026; 5/20,480), and two C. elegans datasets (80/10,782; 27/3,984). Baselines include POCO for forecasting, POYO+ for behavior decoding, and CalM; CAPT outperforms them on held-in and held-out pretraining sessions and across the independent transfer datasets. For the embedding analysis, a linear SVM is trained on pooled neuron embeddings over the 20 most abundant NeuroPAL cell-identity labels: CAPT beats POCO within a dataset, and in the harder cross-dataset setting — classifier trained on one C. elegans dataset and tested on the other — it retains substantial accuracy while POCO falls to chance.

#Applications

The frozen-backbone setup is aimed at systems neuroscience labs whose own recordings are far smaller than a pretraining corpus. Attaching lightweight session and neuron embeddings plus a task head lets a group reuse general population dynamics for forecasting future activity, decoding behavioral variables from population state, or extracting embeddings for downstream analysis — including cell-identity work in whole-brain C. elegans imaging, where NeuroPAL labels make anatomical identity checkable. Because the same checkpoint serves multiple tasks and species, it offers an alternative to training a bespoke model for every new dataset.

#Impact

CAPT supplies direct evidence that a single continuous autoregressive checkpoint can act as a reusable backbone for calcium imaging across labs and organisms — a claim the per-dataset convention in this literature had left unexamined. The authors are candid about the boundaries: pretraining uses one mouse dataset, so multi-dataset and multi-species pretraining remains future work; transfer gains are generally smaller than gains on the pretraining distribution, indicating the backbone is still partly shaped by its source distribution; methods including CAPT perform better on trial-aligned recordings than on recordings without clear trial structure; and the mechanism behind cross-species transfer is only partially understood. The work is a preprint, and no public code or model weights accompany it.