MoleVers

Of the 1,644,390 assays in ChEMBL, only 6,113 — 0.37% — contain 100 or more labeled molecules. A medicinal chemist who measures a new endpoint typically walks away with a few dozen numbers, not the thousands that MoleculeNet-style benchmarks assume. Pretrained molecular encoders that look strong on those benchmarks often collapse in that regime, sometimes barely improving on the trivial baseline of predicting the mean of the labels.

MoleVers is a pretrained 3D molecular encoder built for exactly that setting, from the Max Planck Institute for Informatics and MIT CSAIL. Its answer is to split pretraining in two. The first stage learns representations from unlabeled structures through masked atom prediction combined with extreme denoising — coordinate and pairwise-distance denoising at noise scales far larger than the near-equilibrium perturbations usually used. The second stage then refines the same encoder against auxiliary properties that are cheap to compute rather than cheap to measure: quantum-chemical quantities from density functional theory, and pairwise property rankings generated by a large language model. Only after both stages does the single resulting checkpoint get fine-tuned on the handful of real experimental labels a downstream assay provides.

The work is accompanied by MPPW (Molecular Property Prediction in the Wild), a benchmark of 22 ChEMBL-derived assays deliberately chosen to be small, and standardized so that competing pretraining strategies are compared on identical pretraining corpora rather than on whoever had the better data.

#Key Features

• Extreme denoising: Training the encoder to recover structures corrupted at large noise scales exposes it to non-equilibrium conformations a conventional denoiser never sees, which the authors find generalizes better to unseen downstream properties.
• Branching encoder: Masked atom prediction is a comparatively easy discrete mapping; coordinate denoising is a hard continuous one. Two encoders of identical architecture take one task each, joined by a pooling-by-multihead-attention aggregator so information still flows between them, and only the masked-prediction branch is carried forward.
• Dynamic noise scale sampling: Rather than fixing the noise level, σ is drawn per sample from a uniform distribution, biasing training toward easier low-noise examples while still supplying hard ones. Raising the maximum σ to 10 improves downstream error consistently; 20 destabilizes training.
• Auxiliary labels instead of scarce ones: HOMO, LUMO, and dipole moment computed with Psi4 give a supervised second stage without any laboratory work.
• LLM rankings rather than LLM values: GPT-4o is asked which of two molecules scores higher on a property, not what the value is, because it is markedly more reliable at relative than at absolute predictions.

#Technical Details

Both encoders follow the Uni-Mol encoder design — 15 layers, embedding dimension 512, feedforward dimension 2048 — with MLP heads for the masked-prediction and denoising targets. Stage one runs for 1 million iterations at batch size 32 with a 0.15 masking ratio over 1 million unlabeled molecules sampled from GDB-17; stage two trains 50 epochs on a 130,000 molecule subset carrying the DFT auxiliary labels, using Adam at a learning rate of 1e-4. All experiments ran on a single NVIDIA Quadro RTX 8000. On the 22 MPPW assays — 38 to 123 molecules each, evaluated over three 50:50 train/test splits — MoleVers gives the lowest mean absolute error on 18 and the second lowest on the remaining four, a 13.67% average MAE reduction and a 133.09% average R² improvement over the strongest baseline (mean R² 0.3241 versus 0.1212 for Uni-Mol, mean Kendall τ 0.4482 versus 0.3751). It also records the lowest MAE on the large QM7, QM8, and QM9 regression sets in MoleculeNet.

#Applications

The MPPW endpoints are the kind of measurement medicinal chemistry projects actually generate: partition coefficients and pKa, brain-to-blood partition, antifungal and antimycobacterial activity, cytotoxicity in specific cell lines, plasma half-life, antioxidant capacity. A group that has run one assay on 50 compounds can fine-tune the shared checkpoint on those labels and rank the next batch before synthesizing it, which is the setting in which ranking quality matters more than absolute accuracy.

#Impact

MoleVers' more durable contribution may be the framing: it makes the case that molecular pretraining should be judged on assay-scale data, and supplies a standardized benchmark for doing so. The auxiliary-label idea also generalizes — any property that is cheap to compute and correlated with the target can substitute for labels that are not. The code, released as a modification of the Uni-Mol codebase and requiring the Uni-Core stack under slurm, is public, as is the processed MPPW data; the pretrained checkpoint itself has not yet been distributed, so reproducing the transfer results currently means re-running both pretraining stages. The work was presented as a poster at the Generative AI for Biology workshop at ICML 2025 and has not been through main-conference or journal peer review.