SPARKS

Neuropixels probes and two-photon calcium imaging routinely deliver the activity of hundreds to thousands of neurons at once, and the standard way to make sense of that is to compress it into a handful of latent dimensions tracking behaviour or stimuli. The methods that decode best tend to be black boxes, their learned parameters saying nothing about the circuit that produced the data; the interpretable ones tend to decode poorly.

SPARKS — Sequential Predictive Autoencoder for the Representation of spiKing Signals — was built by the Iacaruso lab at The Francis Crick Institute, with collaborators at King's College London, to collapse that trade-off. It is a variational autoencoder whose encoder replaces conventional dot-product attention with a Hebbian attention layer: instead of learned query and key projections, the attention weight between two neurons is computed from spike-timing-dependent plasticity traces over their actual spike times, so attention is produced by the mechanism thought to strengthen real synapses. Training uses a predictive-coding criterion — encode the recent past, reconstruct the near future — which forces the latent trajectory to be temporally coherent.

Many neural-signal foundation models work on spatially aggregated recordings such as fMRI — BrainLM, for instance. SPARKS instead operates at single-spike resolution and treats its attention coefficients as a scientific readout: applied to mouse visual cortex, they recover the known functional hierarchy of visual areas as inferred effective connectivity.

#Key Features

• Hebbian attention over spike times: Pre- and post-synaptic eligibility traces decay exponentially and combine into a pairwise attention factor analogous to the query-key product, computed directly from single-spike timing and firing order rather than from learned projections.
• Predictive-coding objective: The autoencoder reconstructs a future window of activity from a past window — tau_p past and tau_f future time bins — enforcing temporal coherence in the latent embedding.
• Interpretable coefficients: Because the attention weights are per-neuron-pair plasticity parameters, they can be read as effective connectivity, recovering the functional hierarchy of the mouse visual cortex.
• Cross-session and cross-animal embeddings: A separate Hebbian attention block per recording feeds a shared encoder and decoder, embedding multiple sessions and animals in one latent space and letting a pretrained model adapt to a new session in roughly ten fine-tuning epochs with no loss of accuracy against training it from scratch.
• Two recording modalities: Distinct attention layers handle binned spike trains and calcium fluorescence, so the same framework applies to electrophysiology and imaging data with minimal preprocessing.

#Technical Details

The encoder stacks the Hebbian attention layer with optional conventional self-attention blocks and a projection head onto a Gaussian latent; the decoder is any PyTorch module, defaulting to a single-layer MLP. Model size is set by the recording rather than fixed, the constructor taking neurons per session, an embedding dimension, and a latent dimension. Published configurations range from a 3-dimensional latent with a 64-dimensional embedding for predicting monkey hand position, to a 64-dimensional latent with a 128-dimensional embedding for jointly embedding multiple mouse visual cortex sessions. Evaluations span monkey somatosensory area 2 during reaching (DANDI dataset 000127), Allen Brain Observatory Visual Coding Neuropixels recordings of natural movies and drifting gratings, and medial entorhinal cortex recordings, with the authors reporting state-of-the-art predictive performance across the electrophysiology and calcium-imaging datasets tested.

The implementation is released as the PyTorch package sparks-ai under GPL-3.0, with demo notebooks, an online training mode that avoids full backpropagation through time, softly bounded STDP for stability on long recordings, and beta sliding-window attention for large recordings. No pretrained checkpoints are distributed — users train on their own data — and the paper remains a bioRxiv preprint under a non-commercial, no-derivatives license.

#Applications

Systems neuroscience labs are the direct audience: SPARKS turns a session of Neuropixels or calcium-imaging data into a low-dimensional trajectory that can be plotted against behaviour, used to decode continuous variables such as hand position or discrete ones such as reach direction, or explored without labels to surface structure like oscillatory dynamics in entorhinal recordings. The attention weights add a second use: estimating functional interactions between recorded neurons and areas from the same fit that produced the decoder. Because one model can span sessions and animals, it supports pooled analyses across a cohort rather than a separate fit per recording.

#Impact

SPARKS argues that biological plausibility and predictive performance need not be opposed: a plasticity rule can serve as the attention mechanism of a modern sequence model and still yield parameters a neuroscientist can interpret. Its reach is currently bounded — the work is a preprint awaiting peer review, evaluation comes from one lab, and no pretrained weights are published. The software is already usable, installable from PyPI with worked notebooks for the three published analyses, giving labs an alternative to black-box latent-variable models when the goal is to understand a circuit rather than only decode it.