Loading…
Self-supervised molecular graph model contrastively pretrained on ~10M unlabeled PubChem molecules, then fine-tuned for property prediction.
No providers recorded yet. Browse all providers
ClinTox has 1,478 molecules. BACE has 1,513. SIDER has 1,427. These are the labeled datasets a medicinal chemist has for toxicity and target-binding questions, and the chemical space they must generalize over runs to something like 10^60 drug-like structures. A graph neural network trained from scratch on 1,500 molecules learns those scaffolds and little else, which is why supervised molecular models degrade sharply on scaffold-split test sets.
MolCLR moves representation learning off the labels entirely. Roughly 10 million unlabeled SMILES strings from PubChem become molecule graphs, and each graph is perturbed twice into two correlated views. A normalized temperature-scaled cross-entropy (NT-Xent) objective pulls the two views of one molecule together while pushing apart views of different molecules in the batch. What makes the framework chemical rather than generic is the augmentation: instead of arbitrary node or edge dropout, MolCLR masks atoms with a learned mask token, deletes bonds, and removes connected subgraphs grown outward from a random origin atom. Each corresponds to something a chemist would recognize — substituting a group, breaking a bond in a reaction, matching a partial scaffold — so the encoder is pressured to encode functional groups and motifs rather than incidental topology.
Yuyang Wang, Jianren Wang, Zhonglin Cao and Amir Barati Farimani developed it at Carnegie Mellon University, posting a preprint in February 2021 and publishing in Nature Machine Intelligence in March 2022. It is an early demonstration that the SimCLR recipe transfers from images to molecular graphs, and became the standard graph-side pretraining baseline much as MoLFormer-XL did on the SMILES string side.
The encoder is a five-layer message-passing GNN with 300-dimensional hidden features and edge features for bond type and direction, followed by mean pooling and a linear map to a 512-dimensional molecular representation; a one-hidden-layer MLP head projects that into the 256-dimensional space where NT-Xent is applied. Pretraining runs 50 epochs at batch size 512 with Adam, a 5 × 10⁻⁴ learning rate and cosine decay after ten warm-up epochs, over the ~10M-molecule PubChem SMILES set assembled for ChemBERTa. The released GIN checkpoint holds roughly 2.4 million parameters, small enough to fine-tune on a single consumer GPU.
Evaluation covers 13 MoleculeNet datasets — 44 binary classification and 24 regression tasks — under an 80/10/10 scaffold split (QM9 uses a random split, following prior work). The GIN variant reaches ROC-AUC of 93.2 ± 1.7 on ClinTox, 89.0 ± 0.3 on BACE and 88.6 ± 2.2 on MUV, and RMSE of 1.11 ± 0.01 on ESOL and 0.65 ± 0.08 on Lipo. Against other pretraining strategies it leads on five of seven classification benchmarks with a 4.0% average improvement, and on five of six regression benchmarks; pretraining is worth 12.4% averaged classification improvement for GCN and 16.8% for GIN over the same architectures trained supervised. On the quantum-mechanical targets QM7 and QM9 it does not match SchNet or MGCN, which consume 3D coordinates a 2D molecule graph does not carry.
The practical use is transfer to assays with a few hundred to a few thousand labeled compounds — blood-brain-barrier penetration, hepatotoxicity and adverse-reaction panels, solubility and lipophilicity regression, single-target inhibitor activity — where the pretrained encoder supplies a chemistry prior the assay itself cannot. The embedding is meaningful without fine-tuning, so it also supports virtual screening by similarity search, and the augmentation operators are useful on their own to anyone training a graph model on molecules.
MolCLR is now a routine comparison row rather than a headline result: CheMeleon, SMI-Editor and CL-MFAP all benchmark against it, and later contrastive frameworks build on its augmentation set directly. Its limits are equally instructive: the representation is purely 2D-topological, so conformer-dependent and quantum properties stay out of reach, and MoleculeNet scaffold-split numbers are sensitive to splitting and hyperparameter search, which makes small margins between pretraining methods hard to read. Code and both pretrained checkpoints ship in the authors' repository under an MIT license, a large part of why the model became a default baseline.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.