NExT-Mol

Generative models for 3D molecules face a dilemma. Diffusion over atomic coordinates handles continuous geometry naturally, which is why it dominates the field, but it provides no guarantee that the resulting atom-and-bond arrangement is a chemically valid molecule, and it can only be trained on the comparatively small datasets that carry 3D conformers. One-dimensional string language models have the opposite profile: trained on SELFIES they emit 100% valid molecules and can consume billion-scale molecule databases, but they say nothing about geometry.

NExT-Mol, from researchers at the National University of Singapore with collaborators at the University of Science and Technology of China, the Chinese University of Hong Kong, and Hokkaido University, takes both. It pairs MoLlama, an extensively pretrained SELFIES language model, with DMT, a diffusion module that predicts 3D conformers, and generates a molecule by writing its 1D string first and then folding it into coordinates. Validity comes free from the SELFIES grammar; geometry comes from the diffusion stage.

The third component is what makes the pairing more than a pipeline. Rather than treating the two stages as independent, the authors transfer MoLlama's learned 1D representations into DMT through a cross-modal projector, so the geometry predictor benefits from chemical knowledge absorbed during large-scale 1D pretraining. The work was presented at ICLR 2025, and the authors describe the combined system as a foundation model.

#Key Features

• Validity by construction: Because generation happens in SELFIES space, every sampled string decodes to a chemically valid molecule, removing the post-hoc filtering that coordinate-space generators require.
• Standalone conformer prediction: DMT predicts 3D conformers for arbitrary input molecules, so the module is useful independently of the generative pipeline.
• 1D-to-3D transfer learning: A cross-modal projector, fed by a bidirectional attention layer that compensates for MoLlama's causal masking, injects pretrained 1D representations into the diffusion model.
• Property-conditioned generation: Conditioning DMT on a target property value steers conformer prediction toward molecules with the requested quantum-chemical characteristics.
• Standard tooling: MoLlama is a Llama-architecture checkpoint on Hugging Face, loadable with LlamaForCausalLM.from_pretrained and no custom code.

#Technical Details

MoLlama is a 960M-parameter decoder-only language model following the Llama 2 architecture, pretrained on 1.8 billion molecules from ZINC-15 represented as randomized SELFIES; randomization exploits the fact that one molecule admits many valid strings depending on graph traversal order. DMT comes in two sizes, DMT-B at 55 million parameters and DMT-L at 150 million. On de novo 3D generation over GEOM-DRUGS, NExT-Mol improves 3D Fréchet ChemNet Distance by 26% relative to prior methods, with gains in geometric similarity and stability; on conditional 3D generation over QM9-2014 it averages a 13% relative gain. Ablations show that adding MoLlama representations improves DMT's conformer accuracy, including on scaffold-split evaluations that test generalization to unseen molecular cores. Pretrained MoLlama weights are public on Hugging Face; the repository states that open-sourcing of the remaining components is still in progress, and neither the code nor the weights declare a license.

#Applications

The system serves structure-based drug discovery and materials design wherever 3D molecular structures are needed at scale — generating conformationally realistic virtual libraries, or supplying starting geometries for docking and free-energy calculations without invoking a cheminformatics conformer generator. DMT alone is useful to anyone who already has molecules and needs conformers, which is a common bottleneck in screening pipelines. Property-conditioned sampling targets the narrower case of designing molecules toward specified electronic or thermodynamic properties.

#Impact

NExT-Mol's contribution is architectural strategy rather than a new generative mechanism: it shows that the enormous 1D molecule corpora, previously usable only by string models, can be routed into 3D generation through representation transfer, and that a language-model front end removes the validity problem that coordinate diffusion has been patching over. The ICLR 2025 acceptance and public MoLlama checkpoint have made it a reference point for hybrid 1D/3D molecular generation. The main practical friction is release status: parts of the codebase remain unpublished, and neither the released code nor the MoLlama checkpoint carries a license, which constrains reuse in commercial or downstream open-source work. The paper is the exception — the ICLR 2025 camera-ready on OpenReview is CC BY 4.0, while the arXiv posting keeps arXiv's default non-exclusive distribution terms.