Uni-3DAR

Three-dimensional structure is the common substrate of a great deal of science — drug-like molecules, protein binding pockets, polymer chains, inorganic crystals — yet the models that handle each of these are almost entirely separate codebases with separate representations. Uni-3DAR, from DP Technology with the AI for Science Institute and Peking University, is an argument that the fragmentation is an artifact of tokenization rather than of the science: given a representation that scales gracefully from an angstrom to a meter, one autoregressive model can cover all of it.

The representation is an octree. Space is recursively subdivided, and only cells that actually contain atoms are partitioned further, so the token budget follows the geometry rather than the bounding box. A structure that would need 32³ = 262,144 tokens as a dense voxel grid is described in hundreds of octree tokens, and deeper trees extend the same scheme to proteins with thousands of atoms. Each occupied leaf then carries a fine-grained token encoding the local atomic detail — a "3D patch" — so compression does not cost structural resolution.

Because generation and understanding both reduce to predicting the next token in this sequence, one decoder-only transformer handles unconditional molecule generation, crystal structure prediction, molecular docking, pocket classification, and property prediction, distinguished only by which tokens the sequence contains. The same models that generate molecules are the ones fine-tuned for atom-level prediction.

#Key Features

• Octree tokenizer: A coarse-to-fine spatial subdivision turns a sparse 3D structure into a compact 1D token sequence, spending tokens only where matter is.
• Two-level subtree compression: Merging two octree levels into a single token shortens the sequence by up to 8x, at the cost of token positions that shift dynamically.
• Masked next-token prediction: An interleaved mask token supplies the position of the token about to be generated, restoring accurate positional modeling under compression; ablations attribute a substantial share of the model's accuracy to it.
• Generation and understanding in one framework: Task-specific tokens separate objectives without interference, and the paper reports mutual benefit between the two when they are trained together.
• Docking without a scoring model: Binding poses are produced as a third frame following the protein and the initial ligand conformation, and ranked by the cumulative probability of their own generation.

#Technical Details

The network is a standard decoder-only transformer at GPT-2 scale: 12 layers, embedding dimension 768, 12 attention heads of dimension 64, SwiGLU feed-forward blocks, pre-norm RMSNorm, roughly 90M parameters. Token embeddings combine type, in-cell coordinates, octree level, frame index, and absolute 3D position. Macroscopic shapes use a VQ-VAE tokenizer in place of the atom-level one.

On QM9 the model reaches 93.7% molecule stability and 98.0% validity, exceeding prior diffusion baselines and UniGEM, which additionally uses molecular property labels during training. On PDBbind2020 docking it improves top-1 success over SurfDock, from 40.96% to 44.75% for poses under 1 Å RMSD and from 68.41% to 69.06% under 2 Å, with median RMSD falling from 1.18 Å to 1.08 Å. Pocket prediction on a CASF-2016, PDBbind and MOAD composite matches or exceeds Vabs-Net using geometry alone, without ESM embeddings or solvent-accessible surface area. After pretraining it ranks in the top two on 8 of 10 small-molecule property tasks and 7 of 8 homopolymer tasks. Across generative benchmarks it reports gains of up to 256% relative to diffusion baselines while running about 21.8x faster at inference.

#Applications

For structure-based drug discovery, Uni-3DAR offers pose prediction and pocket identification from raw geometry, removing the hand-engineered features and separate scoring models that most docking pipelines carry. Its 3D molecular generation is directly usable for de novo design where conformational realism matters, and the pretrained encoders transfer to ADMET-style property prediction for both small molecules and polymers. Materials groups get de novo crystal generation, crystal structure prediction, and PXRD-guided solution from the same codebase.

#Impact

Uni-3DAR is a preprint. Its practical claim is that autoregressive modeling over compressed spatial tokens is competitive with — and considerably faster than — diffusion on 3D structure tasks that diffusion has dominated, which matters for iteration speed in virtual screening. The authors are explicit about the main gap: they train separate models per task rather than one jointly pretrained model over heterogeneous 3D data, so the unified framework is demonstrated but the unified foundation model is not yet built. Its openness splits between artifacts and documentation. The code is MIT-licensed, as are the seven published checkpoints covering molecule generation, crystal tasks, and molecular and protein pretraining, so both are free to reuse and modify. The written record is not: both READMEs are install-and-run instructions carrying no results, leaving the arXiv paper as the sole source of every benchmark number and stated limitation, and the authors posted it under CC BY-NC-ND 4.0 — free to read and cite, but not to adapt or redistribute in modified form. The same groups' Uni-Mol3 applies related 3D tokenization to organic reactions.