SE3-BiLingoMol

Structure-based drug design asks a generative model to satisfy three criteria at once: a chemically sensible 2D topology, a physically valid 3D conformation, and a binding pose that genuinely complements the target pocket. The field has split along architectural lines. Graph neural networks handle geometry well, because SE(3)-equivariance is straightforward to build into them, but they frequently emit molecules with implausible fused-ring systems and low drug-likeness. Chemical language models produce clean, drug-like topologies, but handle continuous atomic coordinates poorly — usually by discretizing them into tokens — and their autoregressive sampling accumulates conformational error atom by atom.

SE3-BiLingoMol, developed by StoneWise with collaborators at Peking University, the Institute of Physics of the Chinese Academy of Sciences, and Capital Medical University, is a chemical language model built to close that gap. It retains a Transformer backbone but swaps in Geometric Algebra Transformer (GATr) layers, which represent 3D coordinates as multivectors in projective geometric algebra and process them equivariantly. A separate bidirectional attention module then relaxes the generated conformation after autoregressive sampling, acting as an intrinsic error-correction step rather than a rule-based post-processing fix.

The model was released as a bioRxiv preprint in September 2025, paired with a prospective medicinal chemistry campaign: an iterative human-AI workflow against hematopoietic progenitor kinase 1 (HPK1), a negative regulator of T-cell receptor signaling and a demanding cancer immunotherapy target.

#Key Features

• Equivariant handling of continuous coordinates: GATr layers embed atomic positions as 16-dimensional multivectors and predict them by regression rather than classification over discretized bins, yielding smooth bond-length distributions that match experimental structures.
• Bidirectional self-refinement: A learnable module gives the model a global view of the partially built ligand in its pocket, correcting drift from autoregressive sampling. An ablation raises conformation validity from 66.6% to 85.2%.
• Dual-channel perception: Scalar attributes (atom and bond types, pocket physicochemistry) and vector features (coordinates) are encoded separately and allowed to interact, so topology and geometry are learned jointly.
• Two generation modes: The same model performs de novo design from an empty pocket and substructure-guided optimization, growing a molecule from a fixed fragment.
• Fast sampling: KV caching and multi-query attention produce roughly 500 valid molecules in 90 seconds, one to two orders of magnitude faster than the baselines tested.

#Technical Details

SE3-BiLingoMol is an encoder-decoder architecture: a pocket encoder ingests residue-atom positions plus scalar features (hydrogen-bond donor/acceptor flags, hydrophobicity, aromaticity, formal charge, residue type, and labels marking atoms prone to non-covalent interactions), and a ligand decoder autoregressively emits tokens in the fragment-SMILES (FSMILES) ring-first representation together with continuous coordinates. The GATr backbone is E(3)-equivariant; the model explicitly breaks mirror symmetry so chirality is preserved under SE(3). Training used 53,871 protein-ligand complexes curated from the PDB under a PDBbind-style protocol, with Foldseek clustering at a TM-score cutoff of 0.5 removing 10,224 structures homologous to the evaluation targets. Benchmarked on 102 DUD-E targets against Pocket2Mol, TargetDiff, Lingo3DMol, PocketFlow, PMDM, and MolCraft, it led on 83.4% drug-like molecules (QED > 0.3, SAS < 5), 87.1% passing the HEAD local-conformation check, a median PoseCheck strain energy of 52 kcal/mol, mean pocket occupancy 0.76, PLIF recovery of 35.3%, and 27.2% of poses scoring better in place than after redocking.

#Applications

The intended users are structure-based drug discovery teams that need pocket-conditioned ideation at scale. In the HPK1 campaign, a pocket defined from PDB 7KAC seeded generation of 1.2 million candidates, filtered by binding score, electron-density complementarity, torsional-energy validity, and drug-likeness, then clustered on Bemis-Murcko scaffolds to 200 for chemist review. That surfaced a novel tetracyclic core; the resulting Cmpd. 2 improved enzymatic potency roughly 100-fold over the starting hit and was confirmed by a 3.2 Å co-crystal structure. A second, fragment-constrained round produced Cmpd. 6, which showed greater than 100-, 80-, 350-, and 300-fold selectivity over GLK, MAP4K5, TNIK, and TBK1 in a 108-kinase panel and 56% and 93% tumor growth inhibition at 1.5 and 5 mg/kg orally in a CT26 model, rising to 89% and 102% combined with PD-1 blockade.

#Impact

SE3-BiLingoMol demonstrates that the language-model branch of 3D molecule generation can match graph-based methods on geometry without giving up its advantage in drug-like topology, and the accompanying campaign is unusual in carrying a generated scaffold through crystallography, pharmacokinetics, and in vivo efficacy rather than stopping at in-silico metrics. StoneWise subsequently released UniLingo3DMol, a related pocket-based generative language model applied to CBL-B. Several caveats apply: the work is a preprint that has not been peer reviewed, the compound data are self-reported industrial results, and source code and partial training data have been deposited only for peer review, with public release promised upon publication — no weights, model card, or data card are currently available. The authors also note that quantum-mechanical conformational accuracy remains out of reach for purely data-driven generation, and that generation is not explicitly guided by novelty or synthetic accessibility objectives.