MolSight

MolSight is a graph-aware vision-language model (VLM) for unified understanding of molecular structure images. It targets a persistent weakness in molecular large language models: while these systems reason fluently over textual representations such as SMILES strings, they struggle to interpret the two-dimensional drawings in which chemistry is actually communicated — the structure diagrams found in patents, journal figures, and reaction schemes. MolSight closes that gap by teaching a general-purpose VLM to see molecular topology, not just pixels.

The model was developed by Wenda Wang, Yihan Tong, Yuwei Hu, and Zhewei Wei at Renmin University of China and released as a preprint in July 2026. Rather than building a narrow optical chemical structure recognition (OCSR) tool that outputs a single SMILES string, MolSight treats molecular image reading as a broad, multi-task problem spanning structure transcription, captioning, physicochemical descriptor estimation, and bioactivity prediction. This positions it alongside molecular LLMs and specialized recognition engines while unifying their capabilities under one multimodal framework.

Its central idea is that accurate molecular understanding requires explicit topological modeling. Two purpose-built modules inject chemical-graph structure and symbolic grounding into an otherwise generic visual backbone, allowing the model to connect what it sees to the underlying bonding pattern of the molecule.

#Key Features

• Molecular Topology Module (MTM): Injects chemical-bond adjacency information into the model's vision tokens through learnable edge prediction, giving the VLM an explicit representation of which atoms are connected rather than relying on pixel patterns alone.
• Molecular Grounding Module (MGM): Aligns visual features with chemical symbolic semantics using cross-modal attention over SVG-derived annotations, tying regions of the image to the atoms and bonds they depict.
• Unified multi-task interface: A single model handles SMILES translation, molecular captioning, descriptor estimation, and bioactivity prediction rather than requiring a separate tool per task.
• Parameter-efficient adaptation: Downstream tuning updates only about 90.4M parameters via low-rank adaptation (LoRA, rank 32, α 64), keeping the large backbone frozen and making specialization inexpensive.

#Technical Details

MolSight builds on the Qwen3-VL backbone, using the 8B-parameter variant as its base VLM and layering the topology and grounding modules on top. Stage-one pretraining uses PubChemSTM (249,545 image–text samples), after which the model is fine-tuned per task: PubChem for SMILES translation (95k train / 5k test) and descriptor estimation, MoleculeQA for molecular captioning (49,993 train / 5,786 test), and MolVision for bioactivity prediction (44,112 train / 11,028 test). On SMILES translation it reaches 0.84 accuracy with a 0.97 Tanimoto similarity and 99.7% chemical validity; captioning on MoleculeQA reaches 70.90% total accuracy; descriptor estimation attains an average Pearson correlation of 0.984 with a mean absolute error of 0.49; and bioactivity prediction averages 0.93 accuracy across six tasks. Across these benchmarks the authors report that MolSight outperforms existing VLMs, molecular LLMs, and specialized OCSR tools.

#Applications

MolSight is aimed at cheminformatics and drug-discovery workflows where chemical knowledge is locked inside images. It can transcribe structure diagrams from patents and literature into machine-readable SMILES for database construction, generate natural-language descriptions of depicted molecules, estimate physicochemical descriptors directly from a drawing, and flag likely bioactivity. Medicinal chemists, data-curation teams building molecular corpora, and researchers digitizing legacy chemical documents are the primary beneficiaries, since the model collapses several specialized pipelines into one image-to-knowledge interface.

#Impact

MolSight advances molecular image understanding by demonstrating that explicit graph-topological signals and symbolic grounding materially improve how vision-language models read chemical structures, moving beyond string-only molecular LLMs and single-output OCSR engines toward unified visual reasoning over chemistry. As a July 2026 preprint it awaits peer review, and no public code or model weights had been released at the time of writing, which constrains independent reproduction; its reported gains are in-silico benchmark results rather than experimentally validated predictions. Even so, the topology- and grounding-aware design offers a template for grounding multimodal chemical models in molecular structure.