Retinal encoding models that predict ganglion cell responses to visual stimuli, with pretrained checkpoints across four species and two modalities.
No providers recorded yet. Browse all providers
openretina is a collection of neural encoding models — networks trained to predict, frame by frame, how a retina's ganglion cells fire in response to a video shown to it. The retina is an appealing target because it is a self-contained computational module of the central nervous system, with well-characterized cell types and accessible ex-vivo recordings that make a complete input-output model a realistic goal.
The obstacle has been fragmentation. Decades of retinal recordings sit in lab-specific formats behind lab-specific codebases with incompatible evaluation protocols, so a model reported on one dataset cannot be compared to one reported on another. openretina, from the University of Tübingen with collaborators at the University of Göttingen, Sorbonne Université, and the Max Planck Institute for Dynamics and Self-Organization, standardizes the whole stack — data format, architecture interface, training pipeline, and metrics — and ships pretrained checkpoints for every dataset it integrates.
Every model follows the "Core + Readout" convention standard in visual system identification. A Core, shared across all neurons in a session, extracts a spatiotemporal feature representation from the stimulus; a lightweight per-neuron Readout indexes into that feature map and emits a firing rate. The separation is what makes the framework general: a classical linear-nonlinear-Poisson model is the case where the Core is a pass-through, and any vision backbone producing a spatial feature map can serve as a Core.
openretina.insilico module optimizes most exciting inputs (MEIs) for a neuron, generates most discriminative stimuli that drive one cell group while suppressing another, visualizes convolutional and readout weights, and maps response gradients around the MEI.The package is built on PyTorch with PyTorch Lightning and Hydra, so architectures and datasets can be swapped from a config file. Cores in the initial release include spatiotemporally separable 3D convolutional stacks and Gated Recurrent Unit variants; Readouts use a Gaussian spatial mask over the Core's feature map combined with per-channel feature weights. On the Höfling et al. mouse two-photon dataset of roughly 3,000 cells across 67 sessions, a high-resolution GRU core reaches 0.587 correlation against trial-averaged responses and 0.445 FEVE, against 0.293 and 0.029 for an LNP baseline; moving from 18×16 to 72×64 pixel inputs and from greyscale to UV/green improves FEVE while leaving correlation nearly unchanged. Across all five datasets, FEVE for movie-trained models ranges from 0.449 to 0.710 after filtering to cells with explainable variance above 0.15 — a spread the authors attribute to species, modality, stimulus, and number of test repeats rather than to model quality. The models currently implemented are data-driven, with both Core and Readout fit from scratch on stimulus-response pairs for a given dataset; the Core/Readout separation is groundwork for future joint training across datasets rather than a demonstrated cross-dataset transfer result.
The primary users are systems and computational neuroscientists studying retinal computation. A trained encoding model is a differentiable stand-in for the tissue: because gradients flow from response back to stimulus, it can be interrogated in ways an ex-vivo preparation cannot, generating hypotheses about what a cell type encodes that are testable at the rig. The paper demonstrates this by explaining why MEI optimization returns opposite-polarity stimuli for ON-OFF local edge cells: gradient field analysis shows the optimizer starts in unstable regions of a contrast-encoding cell's stimulus space. The standardized pipeline also lowers the entry barrier for machine learning researchers who want to work on neural prediction without first reverse-engineering a neurophysiology data format.
openretina is a preprint and has not yet been peer reviewed. It follows the pattern set by Brain-Score, the Sensorium challenge, and the Algonauts Project — shared benchmarks that let a subfield accumulate results rather than restart them — and applies it to retinal system identification, where no such platform existed. Its cross-dataset comparison makes the field's open problem quantitative: even the best current models leave a large fraction of explainable variance unexplained. The package builds on and credits the Sinz lab's neuralpredictors, is MIT licensed, and is designed for outside dataset contributions, so its usefulness scales with how much of the community's data ends up in it. An online model zoo documents every shipped checkpoint — architecture type, source dataset and paper, expected input tensor shape, size, and the one call that downloads and caches it — with performance figures in the CC BY preprint.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.