BIT (Biomolecular Interaction Transformer)

Pretrained molecular models have generally picked a side. Protein language models learn from sequences, molecular graph and 3D models learn from small molecules, and each grows strong in its own domain while learning nothing about what structure-based drug discovery turns on — how a ligand and a pocket fit together. Pairing two separately pretrained encoders after the fact leaves the interaction itself unmodeled.

BIT, the Biomolecular Interaction Transformer, was released as a preprint in March 2025 by researchers at Zhejiang University and Alibaba Cloud, with collaborators at Nanjing University of Posts and Telecommunications. It treats every input as a set of atoms and pushes small molecules, protein pockets, and protein-ligand cocrystal complexes through one shared Transformer, in 2D graph form, 3D coordinate form, or both. Two mixture-of-experts mechanisms keep the domains distinct: Mixture-of-Domain-Experts swaps the feed-forward block for a molecule expert and a protein expert, and Mixture-of-Structure-Experts gives each domain its own pairwise attention-bias network.

One pretrained checkpoint is then fine-tuned into three shapes — a fusion encoder reading a complex jointly for affinity regression, a dual encoder embedding pockets and ligands separately for retrieval-style screening, and a unimodal encoder for small molecules alone. Where generative models such as PocketXMol propose new molecules for a site, BIT scores and ranks them.

#Key Features

• Mixture-of-Domain-Experts: Each Transformer block keeps one shared attention module but two feed-forward experts, and every atom token goes to the molecule or protein expert by its domain rather than through a trainable gating network.
• Mixture-of-Structure-Experts: Distinct 2D bias experts per domain, plus separate 3D bias parameters for intra- and inter-molecular distances, so contact geometry across an interface is encoded differently from bonded geometry within a molecule.
• Unified denoising pretraining: A single corrupt-then-denoise recipe — perturbing atom coordinates and masking atom types — applies unchanged to unbound molecules, unbound pockets, and bound complexes, letting all three corpora train one backbone together.
• Three downstream configurations: The same weights become a fusion, dual, or unimodal encoder depending on the task, each reached by fine-tuning rather than retraining.
• Screening throughput from precomputed embeddings: In dual-encoder mode pockets and ligands are embedded independently and compared by dot product, screening a billion molecules in under two days on a single V100.

#Technical Details

The backbone extends Transformer-M: 12 layers, hidden size 768, and 32 attention heads, with 2D and 3D structural channels injected as attention bias terms. Pretraining draws on three corpora — the Q-BioLiP database of 967,085 protein-ligand interactions with 3D cocrystal structures (metal-ion and nucleic-acid ligands filtered out), the 3.4 million DFT-equilibrium organic molecules of PCQM4Mv2, and 2 million pockets detected by P2Rank across 0.2 million Protein Data Bank structures. Only the pocket — protein atoms within 5 Å of the ligand — is fed to the model. Training ran 200,000 steps at batch 1,536 (512 molecules, 512 pockets, 512 complexes) on 64 V100 GPUs.

On the PDBbind v2016 core set, fine-tuned BIT reports RMSE 1.175, MAE 0.919, SD 1.166, and Pearson R 0.845, against 1.232 / 0.940 / 1.207 / 0.830 for Transformer-M, and holds up on the external CSAR-HiQ set at RMSE 1.522 and R 0.838. Virtual screening reaches AUC 97.6 with a 0.5% ROC enrichment of 147.76 on DUD-E, and AUC 61.04 with EF0.5% of 10.02 on the harder LIT-PCBA benchmark, ahead of contrastive-pretrained baselines despite no contrastive pretraining objective. On eight MoleculeNet classification tasks it leads on six, a 1.9% relative gain in average ROC-AUC.

#Applications

BIT is aimed at the ranking half of a drug discovery campaign: prioritizing compounds against a target with a known or modeled pocket, estimating affinity for complexes that already have structures, and predicting ADMET-style properties from 2D structure alone. The paper carries this through end to end on the GluN1/GluN3A NMDA receptor, a subtype with no crystal structure and no known small-molecule modulators. Homology modeling and molecular dynamics produced ten receptor conformations, P2Rank supplied the top 100 pockets, dual-encoder BIT narrowed 18 million catalog compounds to 300,000, and a ligand-based fine-tune on 18,678 PubChem records ranked those down to ten assay candidates. Two inhibited below 5 µM, the best at an IC50 of 2.67 µM.

#Impact

The result that matters is the ablation: adding unbound molecules and pockets to a modest complex corpus improves every downstream task, and removing the domain and structure experts hurts binding tasks most. Atom-level knowledge transfers across biochemical domains, and the transfer needs explicit routing to survive. Two limitations bound the work. Every reported number comes from task-specific fine-tuning rather than zero-shot use of the pretrained weights, and although the preprint states that code and pre-trained models will be made publicly available, no repository or checkpoint has been released, so the benchmarks cannot yet be independently reproduced. The paper is a preprint awaiting peer review.