PanoraOnc

Fitting an outcome model for a rare cancer means assembling a cohort of patients like the one in front of you — hundreds of matched cases with recorded therapies and recorded deaths — which a rare tumor, an uncommon alteration, or a hospital without a decade of genomic profiling does not have. PanoraOnc moves that cost upstream: it is pretrained once on 84,131 patients spanning 66 cancer types, then applied to a new cohort using no data from it at all.

Developed by the Michor lab at Dana-Farber Cancer Institute and posted as a medRxiv preprint in August 2026, PanoraOnc is a continued-pretrained vision–language model rather than a purpose-built network. Each patient case is serialized into a JSON-like text record and tokenized, and predictions come back as text tokens rather than through a classification head, so one checkpoint answers response, survival, attribution and counterfactual questions through a single interface. The base model is the general-purpose Gemma3-12B, not a medically tuned derivative such as MedGemma.

What separates this from cohort-specific modeling is where the evaluation happens: the Dana-Farber-pretrained checkpoint is scored zero-shot on Memorial Sloan Kettering and international melanoma cohorts, with pretraining and evaluation sets separated by explicit set subtraction. Its advantage is largest where local training fails, at small sample sizes.

#Key Features

• Zero-shot transfer across institutions: Pretraining on Dana-Farber data alone lifts external zero-shot AUROC to 0.77 on the MSK pan-cancer, pan-treatment cohort, 0.18 above the untrained Gemma 3 base, with no MSK training case.
• One checkpoint, four endpoints: Treatment response and one-, two- and three-year survival are read from the logits at a single target token position and normalized into calibrated probabilities, so no task-specific head is trained.
• Six modalities in one representation: Health-record fields, targeted sequencing panels, whole-exome data, copy-number and tumor-mutational-burden summaries, RNA-seq immune signatures, and spatially resolved immunofluorescence features share one tokenized record.
• Attribution over the raw inputs: Integrated Gradients over token embeddings, aggregated to gene level, rank the features behind each prediction, recovering tumor mutational burden and intratumoral CD8+ infiltration alongside SETD2, DNMT3A, FAT1 and EP300.
• Bias-aware counterfactual scenarios: Outcomes predicted under treatments a patient did not receive are corrected for treatment-assignment bias with targeted maximum likelihood estimation, a doubly robust estimator.
• Explicit break-even points: Crossover cohort sizes are reported — roughly 300 patients for NSCLC immunotherapy, 220 for melanoma, 500 for the MSK pan-cancer cohort — below which zero-shot inference beats training a local model from scratch.

#Technical Details

PanoraOnc adapts Gemma3-12B, a decoder-only transformer paired with a SigLIP vision transformer that stays frozen throughout, since the model consumes image-derived region features rather than raw whole-slide images. All 12 billion base parameters are loaded from the open-weight release; adaptation is Low-Rank Adaptation over a 4-bit quantized Unsloth implementation, in two configurations — r = 8, alpha = 8 and r = 16, alpha = 16 — for task-specific adaptation. Patient records are tokenized with Gemma3's SentencePiece tokenizer into 2,048–4,096 tokens and embedded at dimension 3,840. Ablations covered 4B and 27B bases, and every reported result uses the 12B model.

Pretraining draws on three cohorts: 58,694 Dana-Farber patients across 66 cancer types profiled with OncoPanel v1–v3, 24,950 MSK-CHORD patients profiled with MSK-IMPACT, and 487 melanoma patients pooled from eight international immunotherapy studies, together with a 1,499-patient Dana-Farber sub-cohort carrying multiplex immunofluorescence imaging. Zero-shot AUROC improvements over the base model run 0.08 to 0.18 for the Dana-Farber-pretrained checkpoint and 0.07 to 0.17 once MSK data is added. After fine-tuning on the full Dana-Farber pan-cancer cohort, combining all modalities reaches 0.82 AUROC for two-year survival against 0.77 from clinical features alone.

#Applications

The intended users are translational researchers and molecular tumor boards working where matched cohorts are thin: rare cancers, uncommon alterations, late-line disease, early-phase trials too small for a bespoke survival model. The counterfactual machinery also stratifies a cohort by which treatment category the model ranks highest, which is how the paper flags NSCLC subgroups on chemotherapy whose later-stage EGFR-mutant tumors point toward targeted therapy. A Gradio chat interface, Dr. PanoraOnc, wraps the checkpoints for scenarios. The authors state that the tool is for research use only and is not validated for clinical deployment.

#Impact

Pan-cancer clinico-genomic pretraining, on this evidence, buys the kind of transfer that sequence pretraining bought in molecular biology: a fixed checkpoint carrying useful structure into institutions and tumor types it never saw. The limits sit in the release story. Code is promised on GitHub upon publication, and no repository has appeared. Checkpoints pretrained on Dana-Farber data cannot be shared at all — patient consent and review board constraints forbid it — which makes the variants carrying the headline zero-shot numbers structurally unavailable. The authors describe the MSK-only checkpoint as released and open-weight, but the preprint names no download location for it. Evaluation is retrospective throughout, on observational data with documented treatment-assignment bias.