Llamole

Ask a language model to design a molecule that inhibits both HIV replication and β-secretase 1, weighs around 284 Da, and can be made from reagents you can buy, and it answers by spelling a SMILES string one character at a time. Across fourteen LLMs tested this way — 7B to 70B, in-context or fine-tuned — none matched GraphGA, a plain graph genetic algorithm. Molecules are graphs, and spelling a graph is not the same as constructing one.

Llamole, from the University of Notre Dame, MIT CSAIL and the MIT-IBM Watson AI Lab at IBM Research, splits the job instead. The base LLM keeps the text; dedicated graph models keep the chemistry. Two learnable trigger tokens, <design> and <retro>, mark where the language model should stop writing and delegate. When a trigger fires, query tokens condense everything written so far into a conditioning vector that activates either a Graph Diffusion Transformer, which samples a molecular graph, or a graph neural network, which predicts a reaction template. A contrastively pretrained graph encoder then reads the molecule back into the LLM's embedding space, so text generation resumes with the structure in context rather than a re-serialized string.

The second half of the answer is the synthesis: A* search over an AND-OR tree rooted at the designed molecule, with the base LLM computing the heuristic that ranks which branch to expand, until every leaf is a purchasable compound. This makes Llamole the first multimodal LLM to interleave text and graph generation for molecular design. It was published at ICLR 2025; the same Notre Dame lab has since released DemoDiff and CoMole.

#Key Features

• Interleaved text and graph output: One response mixes prose, generated molecular graphs and reaction steps, so a design rationale and its route arrive together rather than as separate tool calls.
• Trigger-query-prediction control: The LLM decides when to invoke a graph module by emitting a trigger token, then summarizes prior text into query vectors that condition it — no external router or hand-written pipeline.
• A* retrosynthetic planning: Routes are searched over an AND-OR tree with LLM-derived heuristics, the GNN proposing top-50 templates per expansion and search stopping after 300 iterations or 30 seconds.
• Swappable base model: Three LoRA-adapted variants wrap Llama-3.1-8B-Instruct, Qwen2-7B-Instruct and Mistral-7B-Instruct-v0.3 around the same frozen graph modules.
• MolQA benchmark: A purpose-built instruction dataset of graph-text design questions, released with the model, covering drug-like molecules and polymers.

#Technical Details

The graph decoder is a 574M-parameter Graph Diffusion Transformer — 28 layers, hidden size 1024, 16 heads, 4096 MLP width — injecting categorical, continuous and SciBERT text conditions through adaptive layer normalization. The reaction predictor is a 632M-parameter three-layer GIN over 30,124 reaction templates, and the molecule encoder a 43M-parameter five-layer GIN trained with a CLIP-style contrastive loss. All three are pretrained on MolPair — roughly 600K graph-text and 1.6M reaction-text pairs — and frozen during fine-tuning; only the LoRA weights, trigger tokens and projection layers train.

MolQA was built from 3.8M USPTO patent reactions (1976–2016) by depth-first search for routes terminating in Enamine's 1.3M purchasable building blocks, yielding about 139K targets with routes of length 1 to 10: roughly 126K for training and 11K for evaluation (9,986 drug questions, 750 material). Eight properties define controllability — HIV inhibition, blood-brain barrier permeability and BACE1 inhibition for drugs; CO₂, N₂ and O₂ permeability, fractional free volume and thermal conductivity for polymers. The 7–8B variants rank top-three across all twelve metrics, ahead of 70B baselines an order of magnitude larger: validity improves 14.6%, property controllability 32% to 80%, and retrosynthesis success rises from 5.5% to 35% for drugs and 17.9% for polymers.

#Applications

The target use is a design brief stated in ordinary language — property constraints, a structural hint, and a demand that the result be synthesizable — answered with a candidate and a route to make it. That covers medicinal chemistry triage, where synthesizability filters proposals earlier than a downstream planner would, and gas-separation membrane design, where the polymer targets are permeability and free volume rather than binding. The repository ships a Gradio interface, so chemists can drive the model conversationally.

#Impact

Llamole's contribution is architectural: an LLM can act as a controller over frozen domain models, deciding when to hand off and how to condition them, rather than being fine-tuned to emit chemistry as text. MolQA makes the composite task — design plus planning, judged on both — reproducible. The caveats are real. MolQA's property labels come from supervised predictors and LLM-templated text, not experiment, so controllability is measured against in-silico oracles; a 35% retrosynthesis success rate still means most designs finish without a route; and polymer coverage rests on 2,196 monomers. The base LLM weights are third-party releases obtained separately — what this project distributes is the adapters, connectors and pretrained graph modules.