Lingjian

A medical laboratory scientist reading a peripheral blood smear does four things at once: finds the cell worth looking at, names it, says what about its nucleus and cytoplasm justifies that name, and points at where on the slide it sits. Computer vision has traditionally split those apart — a classifier for the label, a detector for the coordinates, nothing at all for the justification. Lingjian collapses them into one generative model: the answer comes back as a single piece of text, with bounding-box coordinates written into the response as integers rather than emitted by a separate detection head.

Lingjian was developed at Jiulongpo District People's Hospital in Chongqing with collaborators across four other Chongqing institutions, by domain-adapting a general instruction-tuned vision-language backbone on a corpus assembled for laboratory microscopy. Its scope is deliberately narrow: peripheral blood, bone marrow, urine sediment, stool, and the parasites and microorganisms seen in those preparations.

That narrowness distinguishes it from the broader medical multimodal models it shares machinery with. BiomedCLIP and PLIP learn contrastive image–text embeddings; LLaVA-Med and Lingshu span radiology, dermatology and histopathology. Lingjian trades breadth for the vocabulary of the laboratory bench, where an error means calling a lymphoblast a myeloblast, or a Plasmodium vivax ring form a falciparum one.

#Key Features

• Four tasks from one set of weights: image comprehension, cell identification, morphology description and target localization, rather than an ensemble of task-specific heads.
• Coordinate textualization: bounding boxes are serialized as absolute integer pixel coordinates inside the generated text, triggered by a /bbox marker on the instruction. Training includes questions about absent targets, so the model can decline to box something.
• Three-stage domain adaptation: a vision-only alignment stage, a caption-plus-grounding stage with the language model unfrozen, and a dialogue stage that mixes back earlier data to limit catastrophic forgetting.
• Externally evaluated as a fixed checkpoint: the headline results come from proficiency-testing items and a prospective reader study excluded from every training stage, with image-level deduplication against the training corpus to check for leakage.
• Balanced across subspecialties: accuracy is near-uniform across blood cells (91.3%), urine sediment (92.0%) and parasites (97.3%), where individual human readers show pronounced specialization.

#Technical Details

Lingjian is built on the Qwen3-VL-8B-Instruct backbone and trained with MS-Swift on four NVIDIA A800 80 GB GPUs. The corpus is over 300,000 public images plus roughly 100,000 proprietary institutional images, alongside about 20.35 million tokens of text from morphology textbooks and open educational material at a Chinese-to-English ratio near 5:1. Long captions and question–answer pairs were generated with Qwen2.5-VL-72B and Qwen2.5-72B, seeded by existing labels and overlaid boxes. Stage 1 trains only the vision tower and projector; stages 2 and 3 unfreeze everything at learning rates of 1e-6 for the vision tower and 1e-5 for the language model. The stage-2 checkpoint is named Lingjian-1.0-Morph, the released stage-3 checkpoint Lingjian-1.0-Chat.

On the National Center for Clinical Laboratories External Quality Assessment series for 2021–2025, Lingjian scored 93.0% against 78.1% for the most senior human cohort under a matched closed-book protocol, with Gemini-3 Pro at 75.3% and GPT-5.1 at 43.7%. It reaches 0.9932 accuracy on the Raabin-WBC official test set, but drops to 0.641 on BMEC erythroid staging, whose staining profile is under-represented in training. Localization sits below dedicated detectors — F1@0.5 of 0.855 on pinworm eggs against a reported 0.995 — because coordinate-text outputs carry no confidence scores. On MPB-394, a 394-image benchmark of 197 rare morphological categories under a two-independent-samples rule, it reaches 45.7%.

#Applications

The intended setting is second review rather than primary diagnosis: a junior reader works a slide, then queries the model about a crop or the whole field. A 120-case multireader multicase study across peripheral blood, bone marrow, urine sediment and stool microscopy, with 35% abnormal cases and no pre-marked regions of interest, measured what that is worth. Sensitivity for abnormal screening rose from 69.7% to 91.7% and accuracy from 79.6% to 90.2%, the gain coming from fewer false negatives; the specificity change from 85.0% to 89.5% was not significant. The same capability supports morphology teaching, quality control, and primary-care laboratories where subspecialty expertise is thin.

#Impact

Lingjian is benchmarked against a national proficiency-testing programme rather than held-out folds of its own cohort, and the authors released the 2021–2025 assessment images and scoring scripts so other models can be measured the same way. The limits are stated plainly in the paper: laboratory-level open-book proficiency scores still exceed 95%, so the model beats individual readers but not a team with references; localization trails specialized detectors; performance degrades on unfamiliar staining; and the terminology is tuned for Chinese practice. The proprietary training images and parts of the pipeline stay behind institutional data-use restrictions, so what is released is a checkpoint and an evaluation harness, not a reproducible training recipe.