3DMolFormer

3DMolFormer is a transformer framework for structure-based drug discovery that handles two normally separate tasks — protein-ligand docking and pocket-aware 3D molecule generation — within one pretrained model. It was developed by Xiuyuan Hu, Yang Zhao, and Hao Zhang at Tsinghua University, Guoqing Liu at Microsoft Research AI for Science, and Can Chen and Xue Liu at McGill University and Mila, and was accepted at ICLR 2025.

Docking and pocket-aware design are duals of one another. Docking asks where a known molecule sits in a known pocket; design asks what molecule should occupy that pocket. Methods for each have developed largely independently, so a generative model typically has no internal notion of binding pose and must call an external docking program to evaluate what it produces. 3DMolFormer exploits the duality directly: because the same model learns docking during pretraining, it can use that ability inside the generation loop.

The second obstacle the paper addresses is representation. A pocket-ligand complex mixes discrete information (atom and residue identities, bond types) with continuous information (3D coordinates), and forcing both through one modality loses precision. 3DMolFormer represents a complex as two parallel sequences — discrete tokens and continuous numbers — read by a dual-channel transformer that keeps each in its native form.

#Key Features

• Dual-channel sequence representation: Discrete tokens and continuous coordinates travel as parallel aligned sequences, so geometry is never quantized into tokens and identity is never forced into a continuous embedding.
• One backbone, two tasks: A single pretrained model is fine-tuned separately for docking and for pocket-conditioned generation, rather than requiring distinct architectures for each.
• Docking-informed generation: Because docking is learned during pretraining, the generation fine-tuning stage can exploit the model's own pose-prediction ability instead of deferring to an external docking engine.
• Reinforcement learning for design: The generation task is fine-tuned with reinforcement learning against binding-affinity objectives, while docking is fine-tuned with supervised learning — matching each task to the signal it actually has.
• Full pipeline released: The repository provides pretraining, docking fine-tuning, and generation fine-tuning scripts, so the entire pretrain-to-deployment path is reproducible.

#Technical Details

3DMolFormer has 92 million parameters: 12 transformer layers with 12 self-attention heads each and an embedding dimension of 768. Pretraining used roughly 228 million samples assembled from public sources — about 209 million small-molecule conformations and 3.2 million protein pockets from the Uni-Mol pretraining corpora, plus about 167,000 pocket-ligand complexes from CrossDocked2020. Docking fine-tuning used PDBbind v2020; pocket-aware generation used the training and test split established by TargetDiff.

On the CASF-2016 docking benchmark (285 complexes), 3DMolFormer placed 84.9% of ligands within 2.0 Å RMSD and 96.4% within 3.0 Å, at an average RMSD of 1.29 Å, against Uni-Mol's 80.4%, 87.0%, and 1.62 Å. On pocket-aware 3D drug design across 100 protein pockets, it reached an average Vina Dock score of -9.48 compared with -8.39 for DecompDiff, and a success rate of 85.3% against a reference-ligand rate of 25.0% and a prior best of 24.5% among generative baselines.

#Applications

3DMolFormer targets hit identification and hit-to-lead work in structure-based drug design, where a team has a protein pocket and needs both to place candidate molecules accurately and to propose new ones. Running docking and generation through one model simplifies the loop: designs can be scored with the same machinery that produced them, without shuttling molecules to a separate docking package between rounds. The docking channel alone is usable as a standalone pose predictor for virtual screening campaigns, and the substantial gap between generated and reference-ligand Vina scores suggests value in expanding chemical series around a known binder.

#Impact

3DMolFormer's acceptance at ICLR 2025 reflects a broader shift toward unified molecular models that treat related structure-based tasks as facets of one learned representation rather than as separate pipelines, and its dual-channel formulation is a clean answer to the discrete-continuous mismatch that recurs across 3D molecular modeling. The reported success rate on pocket-aware design is a large margin over prior generative methods. Two caveats deserve emphasis. Vina Dock scores are computational estimates of affinity, not measurements, and generative models are known to produce molecules that score well while being synthetically impractical, so wet-lab validation remains the real test. The repository also carries no license file, which leaves reuse terms unresolved even though the code and training scripts are public.