LUMEN

A radiologist rarely reads a follow-up chest film in isolation. The prior study goes up beside it, and the report describes what moved: an opacity that cleared, an effusion that grew, a line that was pulled. Nearly every chest-X-ray vision-language model accepts one image at a time, so when it is asked "what has changed compared with the baseline?" it can only describe the film in front of it. LUMEN closes that gap by making the pair of studies, and the interval between them, part of the model's input.

LUMEN is an 8B-parameter vision-language model for chest radiograph question answering, developed at the Sheikh Zayed Institute for Pediatric Surgical Innovation at Children's National Hospital with NVIDIA, Universidad Politécnica de Madrid and George Washington University, and presented at ISBI 2026. A single checkpoint handles three kinds of question: diagnostic questions about one radiograph, difference questions comparing a current study with a prior one, and prognostic questions asking what the chest will look like a stated number of days ahead.

That third task had no supervision to learn from. No instruction dataset asks "what changes can be expected in this patient's chest X-ray in 385 days?", so the authors built one, prompting Llama-3.2-11B-Vision-Instruct to convert the retrospective differences already annotated between two studies into forward-looking answers. This construction is what makes prognostic question answering trainable at all, and it is also what bounds it: the targets are language-model restatements of observed change, not clinician-recorded outcomes.

#Key Features

• Paired-study input: One or two radiographs can be passed in a single turn, with the DICOM time interval between them supplied as text, letting the model reason about progression rather than appearance alone.
• Prognostic question answering: Beyond describing what changed, the model answers forward-looking questions drawn from a bank of 100 templates about how findings are expected to evolve.
• Expert predictions in the prompt: Following the recipe from D-Rax, the group's earlier single-image radiologic assistant, TorchXRayVision classifiers supply predictions for 18 conditions plus patient age, race and view. Probabilities become four verbal confidence levels — no, maybe no, maybe yes, yes — at thresholds of 0.25, 0.5 and 0.75.
• Multi-task training without a diagnostic penalty: Training on temporal and diagnostic instructions together leaves single-image accuracy unchanged, so the temporal capability is additive rather than a trade.
• LLM-graded evaluation: Alongside BLEU-4 and ROUGE-L, responses are scored 1–10 by Llama-3.1-405B, which catches the case where a negated answer scores well on n-gram overlap while being clinically wrong.

#Technical Details

LUMEN fine-tunes the pretrained NVILA-8B visual language model, updating the projection layer, the language model and the vision encoder together for one epoch at a learning rate of 1.5e-5 and a global batch size of 128 on four 80 GB H100 GPUs. Training data comes from MIMIC-CXR and the Medical-Diff-VQA question set, with short factual answers expanded into full sentences by Llama-3.2-11B-Vision-Instruct: 129,231 images carrying 428,995 diagnostic and 43,381 difference questions. Evaluation follows the official Medical-Diff-VQA split with one study per held-out subject — 4,190 images, 13,688 diagnostic and 1,369 difference questions. On difference questions LUMEN reaches 0.375 BLEU-4, 0.656 ROUGE-L and a Llama score of 4.611, against 0.020, 0.206 and 3.275 for the same backbone fine-tuned on diagnostic instructions only. On 3,936 prediction questions it scores 0.095, 0.303 and 4.866. Diagnostic performance holds at 0.694 token recall on open-ended questions and 0.863 accuracy on closed-ended ones, matching the diagnostics-only fine-tune it is compared against.

#Applications

The intended setting is decision support at the reading station, where a model that sees both the current and the prior film can draft the comparison sentence a report needs, flag progression a busy reader might pass over, and answer follow-up questions in dialogue rather than emitting a fixed label set. The prognostic mode is exploratory rather than clinical: it demonstrates that a vision-language model can be trained to answer outcome questions at all, which matters for triage and for follow-up scheduling, but the answers are not calibrated risk estimates.

#Impact

LUMEN is the first chest-X-ray vision-language model trained to answer prognostic questions, extending a line of work — LLaVA-Med, D-Rax, and difference-VQA models before it — from describing images to reasoning across time. The gains on difference questions are large, but the evaluation stays inside the Medical-Diff-VQA partition of MIMIC-CXR, with no external cohort and no zero-shot transfer result, and the prognostic supervision is synthetic. The model consumes only two studies, so longer imaging histories are out of reach. No code or weights have been released; the constructed instruction-tuning dataset is promised pending PhysioNet approval, and both MIMIC-CXR and Medical-Diff-VQA are themselves credentialed-access resources.