Uni-Mol

One molecular graph describes many shapes, and only some of them fit a binding site. Most molecular representation learning encodes a molecule as a SMILES string or a 2D graph, both silent about the geometry that decides whether a ligand binds or how a quantum-mechanical property comes out. Earlier attempts at 3D used coordinates as an auxiliary training signal only: the model could not accept positions at inference and could not emit them at all, leaving binding pose prediction and conformer generation outside the reach of pretrained molecular encoders.

Uni-Mol, released in 2022 by DP Technology with collaborators at Renmin University of China, closes that gap by making 3D coordinates both an input and an output. Its backbone is a Pre-LayerNorm transformer over atoms, with three modifications. Euclidean distances between every atom pair are passed through edge-type-aware Gaussian kernels to give an invariant spatial positional encoding; that pair representation biases self-attention and is itself updated from the multi-head query-key products, an atom-to-pair and pair-to-atom exchange adapted from AlphaFold 2's Evoformer but kept cheap enough that the extra cost is marginal; and an EGNN-style equivariant head converts the pair representation back into per-atom coordinate updates. The model therefore reads geometry invariantly to rotation and translation, and writes it equivariantly.

The framework ships as two separately pretrained networks sharing this architecture: a molecular model trained on 209 million conformations and a pocket model on 3.2 million candidate protein cavities, used alone for single-domain tasks and combined for protein-ligand work. The paper appeared at ICLR 2023.

#Key Features

• Coordinates in and out: the equivariant head predicts atom positions directly, so conformer generation becomes conformer optimization and docking becomes a geometry task rather than a re-ranking of externally generated poses.
• Denoising makes masking non-trivial: because the spatial encoding leaks bond structure, masked atom types would be trivially recoverable. Uniform noise of ±1 Å is applied to 15% of coordinates, and pair-distance and coordinate heads must undo it.
• Two encoders, one architecture: the pocket model uses residue-level rather than atom-level masking and a coarse atom dictionary, but is otherwise identical to the molecular model.
• Docking as a learned distance matrix: emitting complex coordinates end to end proved sensitive to the starting pose, so a four-layer encoder over the concatenated ligand and pocket representations predicts all cross pair distances, and a differential evolution search optimizes the pose against that scoring function.

#Technical Details

Each encoder has 15 layers, embedding dimension 512, feed-forward dimension 2048 and 64 attention heads, giving roughly 48 million parameters. Both were pretrained for up to one million steps at batch size 128, combining a cross-entropy atom loss with smooth-L1 coordinate and pair-distance losses. The molecular corpus starts from about 19 million deduplicated molecules from purchasable catalogs, ZINC and ChEMBL; RDKit ETKDG with MMFF optimization generates ten conformers each plus a 2D fallback, yielding 209 million conformations. The pocket corpus comes from roughly 180,000 PDB structures, repaired and run through Fpocket, filtered to cavities contacting 10 to 25 residues: 3,291,739 in all. On MoleculeNet under scaffold splits, Uni-Mol was best on 14 of 15 datasets, with the largest margins on quantum and solubility endpoints (QM7 MAE 41.8 against 58.9 for GEM); it lost on SIDER, where many peptides and natural products fail conformer generation and fall back to 2D graphs. On CASF-2016 it ranked first in docking power with a 91.6% top-1 success rate against AutoDock Vina's 90.2%, and placed 88.07% of poses within 2 Å RMSD against 65.26% for the best conventional baseline. On GEOM-QM9 and GEOM-Drugs it led both coverage and matching metrics.

#Applications

The molecular encoder is finetuned for ADMET and physicochemical property prediction from small assay sets, the regime that motivated molecular pretraining. The paired encoders support structure-based work: predicting where a ligand sits in a known pocket, generating and refining conformer ensembles, and scoring candidate pockets for druggability, where the model reached 0.946 accuracy on the 113-sample NRDLD benchmark. Medicinal chemists and computational screening groups are the direct audience; the authors note that GEOM measures vacuum ensembles rather than bioactive conformations.

#Impact

Uni-Mol became the default pretrained 3D encoder for molecules and pockets, and much of the catalog reuses it as a frozen component rather than reimplementing it: Uni-Clip takes it as a ligand conformation encoder, DTMol builds on its molecule and pocket encoders, and MoleVers follows its encoder design. It also anchors a family from the same lab — Uni-Mol+ for quantum property prediction, Uni-Mol2 at larger scale, Uni-Mol Docking V2, and the later Uni-Mol3 reaction model and Uni-3DAR autoregressive generator. Code, both pretraining checkpoints, the finetuned task checkpoints and the full pretraining corpora are public under the MIT license, which is why the architecture recurs so widely as a baseline and a starting point.