S23D

A molecule is not a single shape. Its behaviour in a binding pocket, its measured properties, and its free energy all depend on an ensemble of accessible low-energy conformations, so generating that ensemble from a 2D structure is a routine step in computational drug discovery. Most published generators build the geometric symmetries of 3D space directly into the network through SE(3)-equivariant layers. Non-equivariant transformers had emerged as a competitive alternative, but the prevailing assumption was that they need far more parameters to compensate for the missing inductive bias.

S23D — "SMILES to 3D" — is a diffusion model from IBM Research that tests that assumption and finds it depends mostly on positional encoding. Viatcheslav Gurev and Timothy Rumbell replaced the specialized equivariant machinery with a plain transformer backbone and a relative positional encoding tailored to molecular graphs: a negative attention bias that grows linearly with the shortest-path distance between two atoms, with a different slope per attention head. This is the graph analogue of ALiBi, the relative positional encoding widely used in language modeling. With that single change, a 24.8M-parameter model outperforms a 64M-parameter non-equivariant baseline on the GEOM-DRUGS benchmark.

The result matters for the same reason that other ensemble-generating models such as Distributional Graphormer do: shape distributions, not single structures, are what downstream physics and scoring functions actually consume. S23D argues that a conventional transformer stack — the architecture that scales most predictably — is a viable substrate for that job.

#Key Features

• Graph-distance attention bias: Relative position between atoms enters as a negative bias on attention logits proportional to their shortest-path distance in the molecular graph, with per-head slopes, removing the need for equivariant layers.
• Standard transformer backbone: The score network is an ordinary transformer stack in the style used for large language models, split into a graph-encoding subnet and a structural subnet that processes coordinate encodings.
• Parameter efficiency: The 24.8M base model improves on every recall metric of the 64M-parameter MCF-B baseline while being 61% smaller.
• Two-stage hydrogen-free training: Pretraining on heavy-atom-only molecules followed by fine-tuning on complete molecules cuts sequence length and compute during the expensive phase of training.
• Post-hoc chirality correction: Generated conformers with inverted stereochemistry are reflected along one axis, a cheap correction that lifts recall coverage by more than two percentage points.

#Technical Details

S23D is a denoising diffusion model over atomic coordinates using a variance-preserving SDE, with the transformer acting as the score network. Two configurations were trained: S23D-S at 8.6M parameters (one graph-encoder block plus nine structural blocks, hidden size 256, 8 heads) and S23D-B at 24.8M parameters (one plus thirteen blocks, hidden size 384, 12 heads). Training used the GEOM-DRUGS splits introduced by GeoMol — 243,473 training, 30,433 validation, and 1,000 test molecules — with 20 conformers sampled per molecule, run for 100 epochs without hydrogens and then 25 to 35 epochs of fine-tuning on complete molecules. On GEOM-DRUGS at a 0.75 Å threshold, S23D-B-1/13 reaches 84.6% mean recall coverage and 0.412 mean recall AMR without any chirality information, against 84.0% and 0.427 for MCF-B; with chirality correction applied it reaches 87.0% mean coverage and 0.380 mean AMR, ahead of the 242M-parameter MCF-L (84.7%, 0.390) at roughly a tenth of the size. The equivariant ET-Flow model remains ahead on precision metrics.

#Applications

Conformer ensembles feed docking, pharmacophore search, shape-based virtual screening, conformer-dependent property prediction, and the setup of free-energy calculations. A generative model that produces them from a SMILES string in place of a stochastic search removes a bottleneck in high-throughput screening pipelines, and the small parameter count makes it cheap enough to run over large compound libraries. The architecture is also directly reusable: because it is a standard transformer, it inherits the tooling, quantization, and distillation techniques already built for language models.

#Impact

S23D's contribution is a negative result about equivariance combined with a positive one about positional encoding: the parameter penalty attributed to non-equivariant conformer generators is largely an artifact of how graph structure was encoded, not an intrinsic cost. The authors are candid about what is unresolved. Stereochemistry is handled only as a post-hoc correction rather than as a model feature, which they identify as the main limitation; the GEOM benchmark splits are random rather than scaffold-based, so benchmark gains may overstate generalization to unseen chemotypes. No code or trained weights have been released, which limits independent replication and direct use.