MegaMolBART

A medicinal chemist who wants twenty analogs of a promising hit is asking for something a computer finds awkward. SMILES strings are discrete, and editing them a character at a time mostly yields strings that are chemically invalid or that bear no resemblance to the compound you started from. MegaMolBART reframes the request. A denoising sequence-to-sequence transformer is trained to reconstruct corrupted SMILES, which pushes its encoder to place chemically similar molecules near one another; generating analogs then becomes sampling points around the hit's embedding and decoding them back into strings. Run on its own, the same encoder turns any molecule into a fixed-length vector that a downstream property model can consume as a feature.

The model came out of a collaboration between NVIDIA and AstraZeneca, announced in April 2021 and developed on NVIDIA's Cambridge-1 supercomputer in the UK. Its architecture descends from Chemformer, the BART-on-SMILES model from AstraZeneca's molecular AI group. NVIDIA re-implemented the recipe in its NeMo-Megatron framework, re-trained it from scratch on a far larger slice of ZINC-15, and released the resulting checkpoint under Apache 2.0 through the NGC catalog — so the two are related by lineage rather than by shared weights. MegaMolBART was the chemistry entry in the first BioNeMo model collection, later joined by MolMIM and GenMol.

#Key Features

• Encoder and decoder in one checkpoint: The encoder alone supplies embeddings for predictive models; the encoder and decoder together sample novel molecules. Both routes read the same latent space, so a property model and a generator agree on what "similar" means.
• Latent-space analog generation: Rather than mutating a string, the model perturbs a molecule's embedding and decodes the neighborhood, which is how BioNeMo exposes it for molecular optimization and de novo generation.
• Augmented denoising objective: Training corrupts inputs by masking decoder tokens and by randomizing the SMILES written for a given molecule with RDKit, so the encoder learns that several strings denote one compound.
• Downstream workflows shipped with the checkpoint: BioNeMo layers retrosynthesis and physicochemical property prediction on top of the frozen model, with no pretraining rerun required.
• Modest hardware footprint: Inference runs on any NVIDIA GPU with more than 8 GB of memory, putting embedding and sampling within reach of a single workstation.

#Technical Details

MegaMolBART is a seq2seq BART transformer with 8 layers, 4 attention heads, a hidden dimension of 256 and 45M parameters, using pre-norm layer normalization and GELU activations throughout, with dropout of 0.1 during training. Inputs and outputs are SMILES of at most 512 tokens. Pretraining used the ZINC-15 database, filtered to molecules of molecular weight at most 500 daltons, LogP at most 5, reactivity level "reactive" and purchasability "annotated", then capped at 512 characters — a corpus NVIDIA reports as roughly 1.45 billion molecules on the NGC model card and 1.54 billion in the BioNeMo dataset documentation, split 99% / 0.5% / 0.5% into train, validation and test. Training ran with data parallelism across 64 V100 32 GB GPUs for 8 epochs, about 160,000 iterations and roughly 80 wall-clock hours, with a micro batch of 32 molecules per GPU, a Noam schedule peaking at a learning rate of 0.0005 after about 8,000 warmup steps, FusedAdam and a categorical cross-entropy loss. NVIDIA's evaluation uses the MoleculeNet lipophilicity (4,200 measured logD values), FreeSolv (642 hydration free energies) and ESOL (1,128 water solubility measurements) sets, and publishes no headline scores for the released checkpoint.

#Applications

The encoder's embeddings serve as drop-in molecular features for QSAR and ADMET models, replacing hand-designed fingerprints for lipophilicity, solubility and hydration free energy. The generative route supports hit expansion and lead optimization: embed a compound, sample its neighborhood, and decode analogs for triage before any synthesis. Fine-tuning the same checkpoint gives reaction and retrosynthesis models, which is how BioNeMo packages it for computational chemistry teams. Because the weights are Apache 2.0 and small enough to run on one GPU, the model is also a practical baseline for groups building their own chemical language models.

#Impact

MegaMolBART marks the point where generative chemistry moved from academic prototypes to vendor-supported infrastructure: a pharmaceutical company and a hardware vendor co-developed a chemistry language model on a national-scale supercomputer and released the weights permissively. It seeded the chemistry half of BioNeMo and set the pattern NVIDIA's later small-molecule models follow. The limits are real. At 45M parameters it is small next to the chemical language models that followed, NVIDIA published no paper or benchmark table for it, and the differences between the two NGC releases are undocumented. Development has moved on: the GitHub repository is unchanged since 2022 and the model appears only in BioNeMo Framework 1.x, not 2.x. The v1.0 NGC checkpoint was refreshed in November 2024 and remains downloadable, so the artifact is intact even though the line has stopped growing.