EPT (Equivariant Pretrained Transformer)

A protein-ligand complex is one physical system obeying one set of rules, but the machine learning pipelines built to study it are usually two. Small-molecule pretraining treats a molecule as atoms carrying element, charge, and bond type; protein pretraining treats a chain as residues whose internal atoms are largely abstracted away. Models that need both have typically stitched two separately pretrained encoders into a dual tower, as Uni-Mol does. EPT (Equivariant Pretrained Transformer) removes the seam with a single unit of structure that both domains can be written in: the block. For a small molecule a block is one heavy atom plus its bonded hydrogens; for a protein a block is an amino acid residue. Any molecular system — ligand, monomer, protein-protein interface, bound complex — then becomes a sequence of blocks with 3D coordinates, and one model can be pretrained over all of them at once.

EPT was developed at Tsinghua University with collaborators at Renmin University of China and the University of Pisa, and published in Nature Communications in February 2026 after appearing as a preprint in February 2024. It follows earlier work from the same group on generalist equivariant architectures, and shares the cross-domain ambition of AlphaFold 3 while doing a different job: EPT is an encoder finetuned for prediction, not a generative structure predictor.

#Key Features

• Block-level unified representation: Heavy-atom units and amino acid units are the same kind of object to the model, so a ligand, a residue, and a whole binding pocket are encoded by one network with no domain-specific tower.
• Block denoising pretraining: Each block is perturbed as a rigid body — a random translation of its center of mass plus a random rotation about it — and the model recovers the forces and angular momentum that restore the original geometry.
• E(3)-equivariant transformer backbone: Scalar and vector features are initialized by an equivariant message-passing layer, then updated by alternating equivariant self-attention and Geometric Vector Perceptron-style feed-forward layers.
• Geometry-aware attention: Attention logits are modulated by an interatomic distance matrix and by three edge types distinguishing bonded, intra-block, and inter-block neighbors.
• Cross-domain transfer: A variant pretrained only on small molecules still beats a from-scratch model on protein-ligand affinity — structural knowledge moves between domains rather than staying where it was learned.

#Technical Details

The released backbone has six layers, a hidden size of 512 and eight attention heads, exceeding 30M parameters; a ten-layer variant, EPT-10, is also reported. Pretraining used 5.89M entries comprising 261M blocks: the Protein Data Bank supplies 59.4% of blocks (77,814 unique sequences over roughly 600k structural entries), GEOM and PCQM4Mv2 conformations supply 35.8%, and 22,295 PDBBind binding pockets the remaining 4.8%. Property labels were stripped to prevent leakage into downstream evaluation. Training ran 50 epochs on eight NVIDIA A800 GPUs.

On the ATOM3D ligand binding affinity benchmark (3,507/466/490 complexes under 30% and 60% sequence-identity splits), the multi-domain pretrained model records the best RMSE, Pearson and Spearman scores against sequence-based baselines including ProtTrans, structure-based baselines including Holoprot and ProtNet, and pretrained baselines including Uni-Mol and ProFSA. On mutation stability prediction, derived from SKEMPI 2.0, it reaches roughly 0.72 AUROC against 0.70 for SiamDiff. On QM9 the six-layer model averages rank 3.33 across the 12 properties, close behind Frad at 3.17, while EPT-10 leads at 2.33 — the gain with depth indicates the architecture scales.

#Applications

The clearest use is structure-based affinity work: ranking candidate ligands against a target pocket, scoring point mutations at a protein-protein interface, or predicting quantum-chemical properties of a small molecule, each reached by finetuning the same released checkpoint. The authors demonstrate screening end to end. After finetuning on about 110,000 docked PDBBind complexes, the model ranked 1,978 FDA-approved drugs against the SARS-CoV-2 3CL protease, placing nearly all eight marketed anti-COVID-19 drugs inside the top 200. On affinity prediction it outperformed both Glide docking and an ESM-2 plus Uni-Mol dual-tower baseline; on the ranking task it beat the dual-tower baseline and performed on par with Glide. Twelve candidates went on to molecular dynamics, and the peptidyl aldehyde Ac-Leu-Leu-Nle-CHO was tested experimentally, inhibiting the protease with an IC50 of 5.47 μM.

#Impact

EPT's contribution is evidence rather than a leaderboard position: pretraining across domains measurably beats pretraining within one, and the block abstraction is enough to make that pooling work without hand-built bridges between representations. Several limits bound the claim. Downstream use is finetuning, not zero-shot inference. The affinity and stability benchmarks derive from PDBBind and the PDB, which are also in the pretraining corpus; the standard sequence-identity splits control for this, but the evaluation is not on a held-out data source. The retrospective screening benchmark uses only ten decoys per target, and extending the model to RNA-ligand systems depends on vocabulary-based rules for decomposing unseen chemistry into blocks. Code and the pretrained checkpoint are public; the GitHub repository ships no software license, though the authors' Zenodo code archive and the figshare pretraining and downstream datasets are all released under CC BY 4.0.