Loading…
Small-molecule generation over SMILES through a fixed-size latent space dense enough that gradient-free search over it optimizes chemical properties.
Encode a molecule into a continuous space, nudge the code slightly, decode it back, and you should get a close chemical analogue. Encoder-decoder models trained on SMILES routinely fail that test: their latent spaces contain holes, regions that decode to strings no chemist could draw, so sampling around a seed returns nonsense rather than neighbours. Variational auto-encoders regularize the space but trade that flaw for posterior collapse, where the code carries too little information to reconstruct from.
MolMIM answers both with Mutual Information Machine learning. The A-MIM objective maximizes mutual information between a molecule and its latent code while minimizing the code's marginal entropy, so representations stay informative and cluster tightly instead of collapsing onto the prior. The authors add one extension: at each step the posterior's standard deviation is drawn from a uniform distribution and fed back into the encoder, training a single model across the full range of encoding uncertainty. The space that results is dense enough that random perturbation of a code lands on a valid molecule.
Density is what makes the space searchable. A Perceiver encoder compresses variable-length SMILES to a fixed-size code that a Transformer decoder reads back, so a black-box optimizer can treat molecules as plain vectors. The authors demonstrate this with CMA-ES, a gradient-free evolutionary strategy usually used as a weak baseline: run over MolMIM's latent codes and scored by property oracles, it optimizes single and multiple objectives with no weight updates and no property predictor trained in. NVIDIA released the work in 2022 and presented it at the MLDD workshop at ICLR 2023.
MolMIM holds 65.2M parameters. Encoder and decoder each run six layers with a hidden size of 512, eight attention heads and a feed-forward dimension of 2048, and the bottleneck is a single latent token of 512 dimensions — the smallest configuration tested, which also gave the best effective novelty. The distributed checkpoint was trained on roughly 1.54 billion ZINC-15 SMILES filtered to molecular weight at or below 500 Daltons and LogP at or below 5, for just over one epoch on 32 A100 GPUs; maximum input is 128 tokens. In the paper's evaluation MolMIM sampled at 98.7% validity, 100% uniqueness and 95.5% novelty, for 94.2% effective novelty, and the distributed checkpoint reconstructs 99.88% of a 250,000-molecule validation set exactly. On constrained optimization it reached 94.6% success on drug-likeness under a 0.4 Tanimoto similarity constraint, and 97.5% success with 71.1% novelty on the four-way GSK3β, JNK3, drug-likeness and synthetic accessibility objective. Given only 100 CMA-ES iterations it succeeded on 37% of drug-likeness tasks against 16% for CDDD, a model trained with explicit property supervision.
The practical loop is lead optimization: hand the model a hit, sample its latent neighbourhood for an analogue library, then let CMA-ES push those analogues toward a scoring function while a similarity constraint keeps them near the original scaffold. Because the optimization target is an ordinary callable, teams can substitute their own predictors for the public oracles. The embedding and hidden-state endpoints supply fixed-length representations for clustering, similarity search, or downstream property models. NVIDIA serves it as a containerized inference microservice, typically chained with docking and structure prediction models into a screening pipeline.
MolMIM was the first application of Mutual Information Machine learning to molecules, and its larger argument is about where effort belongs: a deliberately naive optimizer matched or beat reinforcement learning policies and genetic algorithms once the latent space it searched was well structured, which points at representation quality rather than search machinery as the lever. The advantage widens under restricted query budgets, which matters where oracle calls are the expensive resource. Contemporary SMILES pretraining such as MoLFormer-XL aimed at property prediction, while NVIDIA's later GenMol pursues controlled generation through masked discrete diffusion over fragment strings. The limits are worth stating plainly: every optimization result is in-silico against oracle functions rather than wet-lab validation, and the paper documents its own penalized LogP results exploiting a known flaw in that oracle. NVIDIA distributes the checkpoint under a community license for research and development only.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.