Loading…
Self-supervised 3D molecular pretraining that corrupts and rebuilds tetrahedral local structures, then transfers to toxicity and affinity prediction.
No providers recorded yet. Browse all providers
A ketone, an ether and a benzene ring are different chemical objects, but to a coordinate denoising pretext task they are all just atoms with Gaussian noise added to their positions. Most 3D molecular pretraining works this way: perturb every atom independently, predict the displacement. Language models chose tokens and vision models chose patches before learning anything; 3D molecular models mostly skipped that choice, and an atom alone is not a chemical unit, so the pretext task teaches geometry without teaching what it means.
LEGO makes the choice explicitly. Its unit is the tetrahedron — a non-terminal atom together with its one-hop neighbours — which is the simplest polyhedron in three dimensions and also the shape that carbon backbones and most functional groups are built from. A breadth-first traversal of the molecular graph cuts the structure into a non-overlapping assembly of these local structures, a fraction of them are corrupted, and the model learns to put them back. The work comes from the Beijing Academy of Artificial Intelligence with the Institute of Automation, Chinese Academy of Sciences. The peer-reviewed version in Pattern Recognition renames the method Tetrahedral Molecular Pretraining (TMP), which is also the name the code repository carries.
LEGO is a pretraining objective rather than a new network: it is applied to an existing Transformer-M encoder, and the released artifact is that encoder after pretraining. That sets it apart from contrastive 3D pretraining such as MolCLR, which uses geometry only to shape a 2D representation, and from coordinate-denoising encoders such as Uni-Mol.
Pretraining runs once on PCQM4Mv2, and the released recipe trains a 12-layer Transformer-M encoder with 768-dimensional hidden and feed-forward layers and 32 attention heads for 1.5 million steps. A sweep sets the perturb ratio to 0.10 and the noise scale to 0.3 Å — larger than prior denoising work uses, which the authors attribute to reconstructing structures rather than individual coordinates. At that ratio roughly 2.5 local structures and 10.8 atoms per molecule are perturbed, and the number of leaf atoms in each perturbed unit stays at 3.3 regardless of ratio.
On MoleculeNet under a scaffold split, the fine-tuned model reports the best result on seven of eight tasks, including 86.4 ROC-AUC on BACE, 96.7 on ClinTox and 85.2 on Tox21. Against the same backbone without pretraining it improves 12 of 16 QM9 and MD17 targets. On ATOM3D-LBA it reaches 1.084 RMSE at the 30% sequence-identity split and 1.001 at 60%, reductions of 17% and 20% over the best prior baselines, though its correlation coefficients rank second. A probing task on frozen embeddings classifies atoms into 50 functional groups at 0.417 AUC-ROC, against 0.383 untrained.
The fine-tuning targets are the ones that gate early drug discovery: blood-brain barrier permeability, toxicity and side-effect liability, aqueous solubility and hydration free energy, and BACE inhibition — the properties a medicinal chemistry team uses to triage a virtual library before synthesis. The protein-ligand results extend this to affinity ranking directly from a complex, without the structural optimisation that docking-derived scoring functions require. The objective itself is portable to other 3D encoders.
LEGO argues that choosing the semantic unit is a design axis for scientific pretraining, not a detail, and it supports that with a transfer the authors themselves label out-of-distribution: an encoder pretrained only on small molecules improves binding affinity prediction on protein-ligand complexes. The limits are stated plainly in the paper. The segmentation produces no deterministic codebook of substructures the way a tokenizer does, and because the traversal starts from a random seed atom the decomposition of a given molecule varies between passes — treated as a regulariser, but it leaves no canonical parse to inspect. Every result is computational, with no wet-lab validation.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.