SpectraLLM

Working out a molecule's structure from its spectra is a daily task in synthetic chemistry, metabolomics, and natural product discovery, and it is one chemists still do largely by hand. Automated tools have mostly gone one of two routes: match a spectrum against a pre-compiled library, which fails on anything not already catalogued, or train a bespoke neural network for a single modality, which throws away the complementary evidence a human expert would combine. Infrared and Raman report vibrational modes, UV-Vis reports electronic transitions, NMR reports the carbon–hydrogen skeleton, and mass spectrometry reports fragmentation — each resolves ambiguities the others leave open.

SpectraLLM collapses that distinction by turning every modality into text. Peak lists, intensities, and acquisition metadata are serialised into natural-language prompts, and a large language model is trained to emit the answer directly as a SMILES string. Because continuous spectra (IR, Raman, UV-Vis, NMR) and discrete spectra (MS) end up in the same token space, one model handles any single modality or any combination of them, and adding a spectrum simply lengthens the prompt. No modality-specific encoders, fragmentation rules, or retrieval components are involved.

The model comes from the Institute of Automation at the Chinese Academy of Sciences with collaborators at Hong Kong Polytechnic University and Tianjin University, and was published at ICLR 2026. It is a different proposition from spectral representation models such as DreaMS, which learn embeddings of tandem mass spectra for annotation and fingerprint prediction; SpectraLLM generates the structure itself.

#Key Features

• One shared token space for five modalities: IR, Raman, UV-Vis, NMR (¹H, ¹³C, HSQC), and MS are all expressed as text prompts, so a single checkpoint serves unimodal and multimodal queries alike.
• Acquisition metadata in the prompt: Ionisation mode, collision energy, solvent, and instrument type are included alongside peaks, letting the model condition on how a spectrum was measured.
• Additive spectral evidence: Fusing modalities improves accuracy monotonically — Tanimoto similarity rises from 0.4151 for NMR alone to 0.4875 when NMR, IR, and MS are supplied together.
• Near-perfect chemical validity: Generated SMILES parse successfully around 99% of the time, against 23–79% for the mass-spectrometry baselines it is compared with.
• Open weights: A merged 32B checkpoint and the LoRA adapter are both published on HuggingFace, with the merged model under Apache 2.0. The code repository carries no license file, and the model card is a stub.

#Technical Details

The backbone is Qwen3-32B, frozen and adapted with Low-Rank Adaptation in each transformer layer; training minimises the standard autoregressive cross-entropy over reference SMILES. The corpus combines four public sources — QM9s (simulated IR, Raman, and UV-Vis for ~134,000 molecules at B3LYP/def2-TZVP), the Multimodal Spectroscopic dataset (simulated IR, MS, and NMR for over 790,000 USPTO-derived molecules), MassSpecGym (230,000 experimental spectra across 29,000 compounds), and MassBank — totalling more than 5.5 million spectra over 943,729 unique molecules. Prompts are capped at 1,024 tokens; decoding uses nucleus sampling at p = 0.7 and temperature 0.4.

Against modality-specific baselines the gains are largest where prior tools were weakest. On the Multimodal Spectroscopic NMR split, SpectraLLM reaches Tanimoto 0.4151 and functional-group recovery 0.7209 versus 0.0433 and 0.1718 for NMR2Struct, cutting maximum common edge substructure distance from 30.69 to 8.31. On QM9s Raman it reaches Tanimoto 0.2500 against 0.1089 for the strongest baseline, and joint IR+Raman+UV-Vis reaches 0.3355 with 0.7934 functional-group recovery. On MassBank it improves functional-group recovery to 0.4539 over DiffMS and Spec2Mol. Zero-shot Qwen3-32B, given the same prompts without fine-tuning, scores Tanimoto 0.0308 on QM9s and 0 on the multimodal set, confirming the capability comes from spectrum–text alignment rather than memorised chemistry.

#Applications

The natural users are analytical chemists identifying unknowns: metabolite annotation in untargeted metabolomics, natural product characterisation, reaction product confirmation, and environmental or forensic screening, all cases where a library match is unavailable and several instruments have already been run on the same sample. Because the interface is a text prompt, partial evidence is handled gracefully — a lab with only IR and ¹H NMR gets a prediction from the same checkpoint a lab with full multimodal data uses.

#Impact

Reframing spectroscopy as a language task is the contribution here, and it buys generality that specialised architectures have struggled to reach. The costs are real: fine-tuning consumed roughly 4.3 million times the compute of a lightweight spectrum-specific baseline, and inference averages 5.78 seconds per sample against milliseconds for those models, which confines it to low-throughput, high-precision work. Absolute structure recovery also remains modest — Tanimoto below 0.5 even in the best multimodal setting — and much of the training signal comes from simulated rather than experimental spectra, so performance on instrument-specific artefacts is bounded by how well those simulations transfer.