Surgical video foundation model pretrained by entropy-maximizing compression on 0.78M unlabeled frames from 2,122 minimally invasive procedures.
No providers recorded yet. Browse all providers
C2E, short for Compress-to-Explore, is a self-supervised visual foundation model for minimally invasive surgery video, developed by the Surgical Artificial Intelligence Laboratory at Massachusetts General Hospital with collaborators at MIT CSAIL and the Department of Surgery at Duke University. Real-time understanding of laparoscopic and endoscopic video underpins intraoperative guidance — recognizing the current phase of an operation, tracking which instrument is acting on which tissue, segmenting anatomy, and flagging lesions — but each of those tasks has historically required its own densely annotated dataset, and surgical annotation is expensive, imbalanced, and constrained by patient privacy.
The model's premise is an information-theoretic one drawn from Kolmogorov complexity: a representation that permits the shortest faithful description of an image has, by construction, retained the image's structure and discarded its redundancy. C2E operationalizes this inside an autoencoder by training a latent-compression transformer against entropy-maximizing decoders, which push the bottleneck to preserve clinically salient detail rather than the low-frequency texture that reconstruction losses typically favor. The result is a single pretrained encoder that transfers across surgical tasks instead of being refit per task.
C2E sits alongside Endo-FM, which learns spatio-temporal representations from unlabeled endoscopy clips, and EndoChat, a multimodal assistant for surgical dialogue. Where Endo-FM emphasizes temporal modeling, C2E is a frame-level encoder whose contribution is the compression objective itself.
The encoder is a ViT-scale transformer with 85.22M parameters, trained inside an autoencoder whose reconstruction objective reduces to a standard masked-autoencoder pipeline once the compression term is expanded; the authors show the compression objective is convex and differentiable. Pretraining data span laparoscopic cholecystectomy, proctocolectomy, rectal and sigmoid resection, gynecologic laparoscopy, radical prostatectomy, and sleeve gastrectomy, assembled from Cholec80, HeiCo, DSAD, LapGyn4, SurgicalActions160, GLENDA, ESAD, hSDB-instrument, and the MGH general-surgery collection, with frames sampled at 1 fps, deduplicated, and filtered by CLIP-based image quality assessment. Frames used in downstream evaluation were removed to prevent leakage. On Cholec80 phase recognition, C2E reaches 92.5% accuracy, edging past SurgFormer (92.4%) and LoViT (91.5%). On CholecSeg8k segmentation it reaches 0.74 IoU, above S-SAM (0.71) and SAM-LoRA (0.65). On PolypDiag it reaches 94.0% accuracy, against 91.3% for EndoSSL and 90.7% for Endo-FM.
The intended setting is intraoperative decision support: a shared encoder that a hospital can adapt to phase monitoring, instrument and tissue tracking, anatomical segmentation, or lesion diagnosis without assembling a new annotated corpus for each. The few-shot results are the operationally relevant ones — surgical AI is most needed for procedures that lack labeled data, and C2E recovers usable phase classifiers from a handful of videos. Groups building surgical video analytics, colonoscopy screening tools, or operative-quality assessment pipelines can use it as a backbone.
C2E demonstrates that an explicitly information-theoretic pretraining objective transfers better across surgical tasks than reconstruction or invariance objectives alone, and its saliency and feature-separation analyses give a mechanistic account of why. The evaluation is broad — four task families, five public benchmarks, and a private MGH corpus — but remains retrospective and offline, with no prospective intraoperative deployment reported. The work is an arXiv preprint and has not been peer reviewed, and neither code nor pretrained weights have been released, so the results cannot currently be independently reproduced and the encoder cannot be reused directly by other groups.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.