Chan Zuckerberg Initiative
A deep learning framework for multi-class protein particle picking in cryo-ET tomograms using a 3D U-Net with automated architecture search via Bayesian optimization.
Octopi — an acronym for Object deteCTion Of ProteIns — is a deep learning framework for multi-class particle picking in cryo-electron tomography (cryo-ET) tomograms developed by the Chan Zuckerberg Initiative. It addresses one of the central computational bottlenecks in cryo-ET structural biology: efficiently localizing and classifying multiple species of protein complexes within the dense, low-contrast environment of an intact cellular tomogram. The tool is available on the CZ Virtual Cells Platform alongside the phantom dataset used to benchmark cryo-ET particle picking algorithms.
Unlike single-class pickers or template-matching approaches that must be run independently for each target, Octopi uses a single 3D U-Net model to simultaneously predict the locations of multiple molecular species in one forward pass. The model is trained on the CZ Imaging Institute phantom dataset — an experimental cryo-ET benchmark dataset with ground truth annotations for six biologically relevant macromolecular complexes — and is fully integrated with copick, the storage-agnostic cryo-ET dataset API developed at the CZ Imaging Institute that provides standardized, programmatic access to tomograms regardless of whether they are stored locally or in the cloud.
A distinctive aspect of Octopi is its automated architecture search capability: rather than requiring users to hand-tune U-Net depth, channel counts, and kernel sizes, the framework uses Bayesian optimization via Optuna to explore the model configuration space and identify near-optimal architectures for a given dataset. This makes Octopi particularly useful for researchers who want to train custom pickers on new cryo-ET datasets without deep expertise in neural network design.
Octopi uses a 3D U-Net architecture with six encoder-decoder levels, selected for its ability to capture both fine-grained local context (individual particle features) and coarser contextual information (neighboring density patterns) through skip connections spanning the full encoder-decoder hierarchy. The default encoder-decoder channel progression follows standard U-Net conventions, and the automated architecture search modifies depth and width multipliers around this baseline. Training patches are extracted from tomographic volumes at a fixed size compatible with GPU memory constraints, and sliding window inference assembles full-tomogram predictions at test time.
The model was developed and validated using the CZII phantom dataset, which comprises experimental cryo-ET tomograms with expert-annotated ground truth positions for six protein complexes. The training set consists of six experimental tomograms (one held out for validation), supplemented by simulated tomograms from the phantom dataset to increase training data diversity. The phantom dataset was designed to test algorithm generalization across complexes of varying size and symmetry — from the highly symmetric apoferritin (24-mer) and virus-like particles to the heterogeneous 80S ribosome — making it a stringent benchmark. The copick-based data loading pipeline is compatible with the standard data formats used across the CZ cryo-ET Data Portal and the CZII Kaggle competition datasets, facilitating transfer to community-standard benchmarks.
Octopi is targeted at structural biologists and cryo-ET practitioners who need automated particle picking for subtomogram averaging pipelines. By outputting multi-class heatmaps rather than simple coordinate lists, Octopi integrates naturally into workflows that proceed from picking through subtomogram extraction, alignment, and averaging to reconstruct high-resolution structures. The Bayesian architecture search makes it practical for users working with novel targets not covered by the pre-trained phantom model: with a modest set of annotated tomograms for the new complex, Octopi can train a customized picker without requiring manual hyperparameter optimization. The copick integration means that picked coordinates can be immediately shared and visualized through the CZ cryo-ET Data Portal ecosystem, supporting collaborative annotation and quality control across research groups.
Octopi represents the CZ Imaging Institute's own baseline contribution to cryo-ET particle picking alongside the competitive models produced during the CZII Kaggle challenge. Its value lies in being an accessible, well-documented, and copick-native picker that prioritizes ease of use and reproducibility over raw benchmark performance. By incorporating automated architecture search, it lowers the technical barrier for cryo-ET practitioners to train custom models for their specific targets. As of early 2026, the model has not yet been the subject of a peer-reviewed publication, and systematic benchmarking against top Kaggle solutions on external datasets beyond the phantom has not been published. Its integration with the CZ Virtual Cells Platform and the broader copick ecosystem positions it as a useful reference implementation for the cryo-ET community.