DrugGPT

Chemical space is effectively unbounded — ZINC20 alone catalogues more than two billion purchasable compounds — but the vocabulary needed to write it down is not. Running byte pair encoding across those compounds converges on just 5,373 tokens, and across the narrower set of ligands already observed bound to a protein, 3,560 suffice. DrugGPT takes that as its premise: if a finite token set spans the chemistry, then proposing a ligand is a sequence-continuation problem, and a decoder-only language model can do it.

What makes the target-conditioning work is the input format. Every training example is a single string — <P> + protein amino acid sequence + <L> + ligand SMILES — so the protein and the small molecule share one vocabulary, one token stream, and one attention window. Asking for ligands against a target therefore needs no second encoder, no docking box, and no per-target retraining: supply the sequence up to the <L> marker and the model writes what comes after. Two variations fall out of the same mechanism. Appending a partial SMILES after <L> pins the opening fragment of every generated molecule, and supplying no input at all lets the model show which proteins it is most inclined to design for.

DrugGPT was developed at the First Affiliated Hospital of Xi'an Jiaotong University, with collaborators at Xi'an Jiaotong University and the Second Affiliated Hospital of Xi'an Jiaotong University, and posted to bioRxiv in June 2023. Its checkpoint is the base that both DrugGen and DrugGen 2 fine-tune. It is unrelated to DrugPT, a drug-perturbation expression model whose name differs by one character.

#Key Features

• Protein and ligand in one sequence: The <P>/<L> framing turns target-conditioned design into ordinary prompting, so any amino acid sequence — pasted directly or read from a FASTA file — is a valid query.
• Ligand prompts for scaffold control: A SMILES fragment supplied after the protein constrains the start of every molecule returned, which lets a medicinal chemist hold a warhead or a core fixed and let the model vary the rest.
• Direct inference with no input: Generating unconditionally exposes the model's learned priors over targets rather than answering a question the user already posed.
• A tokenizer built for the substrate: Merging separate ligand and protein BPE vocabularies cuts an average raw string of 700 characters to roughly 200 tokens, making long protein sequences affordable in context.
• Ligands delivered as 3D structures: Generated SMILES are screened with Open Babel, converted to SDF with MMFF94-optimised coordinates, and named by SHA1 hash, so output drops straight into docking and visualisation.

#Technical Details

DrugGPT is a GPT2LMHeadModel decoder — 12 layers, 12 attention heads, 768-dimensional embeddings, 1,024 positions, and 126.6M parameters — trained from scratch rather than fine-tuned from natural-language GPT-2, on the grounds that SMILES and amino acid strings share little with English text. Its tokenizer merges a 3,560-token ligand vocabulary with a 50,000-token protein vocabulary derived from 1,836,729 protein sequences, giving 53,080 merged tokens. Training data came from ZINC20 and the jglaser/binding_affinity collection, which aggregates BindingDB, PDBbind-cn, BioLIP and BindingMOAD into nearly 1.9 million protein–ligand pairs. Each of the five epochs ran in two stages: ligand-only text first, then protein–ligand pairs split 9:1 for validation, with sequences truncated and padded to 768 tokens. Training used AdamW at a learning rate of 5e-4 with 100 warmup steps and a batch size of 8 on a single NVIDIA RTX 4090, ending at a validation loss of 0.04.

#Applications

The intended user is a chemist with a target and no starting series. Prompted with the BCL-2 sequence and asked for 50 molecules, the model returned 73; constrained by the ligand prompt COc1ccc(cc1)C(=O), it returned 54, all beginning with that fragment. Unconditioned generation of 201 ligands produced 167 aimed at a single protein, identified by BLAST as ENPP2 (autotaxin): against the 2,685 ENPP2 ligands then in BindingDB, 112 of the designs matched known compounds and 55 were absent from the database. Both a command-line entry point and a Google Colab notebook run the model, with batch size, temperature, top-k, top-p and heavy-atom count exposed as flags.

#Impact

DrugGPT's practical significance has been as a starting point: its public checkpoint is the base model behind DrugGen and DrugGen 2, which add reinforcement learning against predicted affinity and disease-ontology conditioning respectively. The preprint's own evidence is modest — it analyses single generation runs to illustrate each inference mode rather than reporting aggregate validity, novelty or affinity metrics on a held-out set, so the ENPP2 numbers characterise one run rather than establishing a benchmark. The released repository carries inference and energy-minimisation scripts only, which makes the work reproducible to run but not to retrain, and wet-lab evaluation is announced in the repository's news feed rather than in a publication.