MarginPath

A breast-conserving resection can read as clear on hematoxylin and eosin and still recur, because much of what the specimen's edge has to say is carried not by tumor cells but by the way collagen around them has been rebuilt. Label-free multiphoton microscopy (MPM) shows that remodeling directly on an unstained section, and eight tumor-associated collagen signatures (TACS) have been described in breast tissue on that contrast — TACS4 through TACS8 sitting at the tumor boundary and invasive front, precisely the zone a margin call turns on. The obstacle is not the optics; pathologists are not trained to read MPM images, and an unfamiliar grayscale field has no route into a signed report.

MarginPath closes that gap by translating rather than replacing. From one unstained section it produces three artifacts a pathologist already knows how to use — a virtual H&E rendering of the same field, a pixel-level probability map of margin status, and a written microscopic description — then composes them into an illustrated report. It was built at Fuzhou University with Fujian Normal University, Peking University, and clinical departments at Fujian Medical University and Xiamen Medical College, and published in Advanced Science in 2026. The same group's ViFIT runs the translation in the opposite direction, from routine H&E to virtual fiber images.

#Key Features

• Intensity inversion before unpaired translation: MPM shows bright structures on a dark field and H&E is the reverse, which leads a conventional CycleGAN to confuse background with tissue. Inverting the MPM pixel values first collapses that gap, letting collagen map to collagen without pixel-registered pairs.
• A nine-class microenvironment vocabulary: Rather than only separating tumor from normal, the classifier labels normal collagen, adipose tissue, ductal-lobular structures, TACS4 through TACS8, and tumor — the shipped checkpoint adds a tenth background index — so every margin call arrives with a stromal composition attached.
• Voting sliding window for a full-resolution heatmap: 512 × 512 patches step at a stride of 128, so each pixel is covered sixteen times; the fraction called tumor becomes its probability, turning a patch classifier into a native-resolution confidence map.
• Deterministic probability-to-prose layer: Class probabilities are thresholded into Trace, Mild, Moderate, and Significant bands and rendered as fixed phrasing before any language model sees them, keeping the numbers out of reach of generation.
• Pathologist-selected regions of interest: The pipeline is deliberately not end-to-end — a pathologist draws the suspicious box on the virtual H&E, and the matched MPM region is pulled from the registered coordinates.

#Technical Details

Three components are trained and shipped as fixed checkpoints. The virtual staining network is a CycleGAN over intensity-inverted MPM input, trained unpaired on 24,070 patches of 512 × 512 pixels under adversarial, cycle-consistency, identity, and SSIM losses. The tumor-microenvironment classifier is a ViT-Base-Patch16-224, roughly 86M parameters, trained on 16,654 pathologist-annotated patches augmented to 20,000. The captioning leg fine-tunes Qwen2-VL-2B for three epochs with the vision encoder frozen, on 276 MPM–text pairs with pathologist-authored descriptions. An off-the-shelf LLM composes the final report from the structured prompts; the paper leaves it unnamed, but the released code calls GPT-4o through the OpenAI API.

Specimens came from 158 invasive breast cancer patients: 132 from Fujian Medical University Union Hospital for training, and 26 from Harbin Medical University Cancer Hospital held out for system-level validation. On 172 held-out regions of interest, BLEU-4 is five times that of LLaVA-Med, though GPT-4o edges ahead on ROUGE-L and term F1 with shorter, more conservative reports. Classification over 1,667 patches and margin prediction over 66 regions beat MedMamba, ResNet, and EMO on accuracy, F1, mIoU, and Dice. Regions the heatmap flagged as ambiguous reached 92.31% accuracy against expert consensus on matched H&E sections.

#Applications

The target setting is margin assessment in breast-conserving surgery. Because MPM runs on an unstained section, that section can afterward go through routine H&E, so the system layers onto the existing workflow instead of displacing it; the authors time a full pass at 11–16 minutes against roughly 30 minutes for conventional frozen section. It also reports a microenvironment composition that a tumor-only margin call cannot, and the group extended it into a question-answering module for clinician and patient communication.

#Impact

MarginPath is narrow by construction: one disease, one modality, one training center, with a human in the loop midway through. Its contribution is architectural: a lightweight perception stack speaks clinical language reliably because the numbers are converted to standardized text before generation begins. The repository ships inference code and links all three checkpoints from Google Drive rather than hosting them, but there is no training script for the classifier, no license file, and only a portion of the imaging data is public; the report stage additionally needs a user-supplied OpenAI key and a one-line fix, since the published script has its API credentials stripped out and does not parse as released. The report layer inherits the hallucination risk of the LLM that composes it, and whether the TACS paradigm transfers to other solid tumors remains untested.