Chan Zuckerberg Initiative
A SAM2-based deep learning framework for vesicle segmentation in cryo-ET tomograms and 2D micrographs, supporting both zero-shot and fine-tuned inference.
SABER — SAM2-Augmented Biological Entity Recognition — is a deep learning segmentation framework developed at the Chan Zuckerberg Imaging Institute (CZII) for identifying vesicles and membrane-bound compartments in cryo-electron tomography (cryo-ET) data. It adapts Meta's Segment Anything Model 2 (SAM2), a foundation model originally designed for promptable segmentation in natural images and videos, to the specialized domain of 3D cryogenic imaging. SABER is available on the CZ Virtual Cells Platform as part of the institute's effort to make AI-powered cryo-ET analysis tools accessible to the broader structural biology community.
Vesicle segmentation is a fundamental step in cryo-ET analysis: synaptic vesicles, endosomes, lysosomes, and other membrane-enclosed compartments must be accurately delineated to quantify their populations, measure size distributions, and study membrane interactions at nanometer resolution in near-native cellular context. Traditional approaches rely on manual annotation or threshold-based methods that struggle with the low signal-to-noise ratio and missing wedge artifacts inherent to cryo-ET data. SABER addresses these limitations by leveraging SAM2's powerful spatially-aware image encoder and memory-based propagation mechanism, treating consecutive tomographic slices analogously to video frames, which allows coherent 3D segmentations to be built from 2D slice-by-slice predictions without full volumetric training labels.
SABER is built on copick, the storage-agnostic cryo-ET dataset API developed at CZII, enabling it to operate directly on any dataset registered in the copick ecosystem — including those hosted on the CZ cryo-ET Data Portal — without requiring custom data loading code.
SABER wraps SAM2's pretrained model weights — specifically the SAM2 image and video encoders from Meta's official release — with a cryo-ET-specific inference pipeline. In the 3D volumetric mode, individual XY slices extracted from a tomographic volume are treated as successive video frames: SAM2's memory attention mechanism maintains temporal coherence across slices, propagating segmentation masks through the Z dimension without requiring per-slice prompts once the model is initialized on the first annotated slice. In the 2D mode, standard promptable SAM2 inference is applied directly to micrograph images or tomographic projections.
The zero-shot morphological filtering pipeline applies post-processing constraints calibrated for typical vesicle sizes (10–200 nm diameter range corresponding to roughly 5–100 voxels at standard cryo-ET sampling rates of 10–20 Å per voxel) and shape regularity to filter out false positives arising from high-contrast membrane density variations and staining artifacts. For fine-tuning, the framework exposes the SAM2 mask decoder for task-specific adaptation while keeping the large image encoder frozen, reducing the compute and data requirements for domain adaptation. The copick-based data pipeline reads tomograms in the OME-Zarr format standard across the CZ cryo-ET Data Portal, ensuring format compatibility with the broader ecosystem.
SABER is designed for structural cell biologists studying membrane trafficking, synapse biology, and organelle biogenesis through cryo-ET. Primary applications include automated quantification of synaptic vesicle pools in neuroscience cryo-ET datasets, mapping of endosomal and lysosomal compartments in cells undergoing endocytosis, and quality control screening of large cryo-ET datasets to identify tomograms with sufficient vesicle content for downstream analysis. The zero-shot capability makes SABER immediately applicable to new datasets without annotation effort, while the fine-tuning pathway allows adaptation to specialized vesicle types (e.g., clathrin-coated vesicles, secretory granules) with modest labeled data. Because outputs are written back into the copick ecosystem, SABER integrates naturally into multi-step cryo-ET analysis pipelines alongside other CZ Imaging Institute tools.
SABER represents an important advance in applying video segmentation foundation models to 3D biological volumetric data, demonstrating that the temporal coherence mechanisms in SAM2 transfer usefully to the slice-by-slice geometry of cryo-ET tomograms. As part of the CZ Virtual Cells Platform, it contributes to CZII's broader strategy of making state-of-the-art AI analysis tools accessible to structural biologists who may not have the computational expertise to develop custom segmentation pipelines. As of early 2026, SABER (v1.0.0) is available on the platform in early access; formal peer-reviewed publication and systematic benchmarking against existing vesicle segmentation methods have not yet been released, a factor users should consider when applying the tool to critical quantitative analyses.