GRAM-DTI

Drug-target interaction (DTI) prediction asks whether a given small molecule binds and acts on a given protein — a question that sits at the front of virtual screening, drug repurposing, and mechanism-of-action studies. Most deep learning approaches frame it narrowly: encode a SMILES string, encode a protein sequence, learn a pairwise scoring function. That discards information already available for both partners, including curated textual descriptions of a molecule's function and taxonomic annotations placing a compound within a pharmacological class.

GRAM-DTI is a pretraining framework that folds four modalities into a single shared representation space: SMILES strings, natural-language molecule descriptions, hierarchical taxonomic annotations (HTA), and protein sequences. Rather than aligning modalities two at a time, it uses a Gramian volume-based contrastive objective that treats the four embeddings of a matched drug-target record as vectors whose spanned volume should collapse — a higher-order criterion generalizing pairwise contrastive learning. Two mechanisms make that alignment practical: gradient-informed adaptive modality dropout, which regulates each modality's contribution so dominant-but-uninformative channels do not drown out complementary ones, and weak supervision from IC50 measurements, which anchors the space to measured interaction strength rather than binary labels alone.

The work comes from the SMILE lab at the University of Texas at Arlington with co-authors at Johnson & Johnson Innovative Medicine, posted to arXiv in September 2025 and accepted at the NeurIPS 2025 Workshop on Multi-modal Foundation Models and Large Language Models for Life Sciences. It trains no new sequence backbones; it composes existing ones — MoLFormer-XL for chemistry and ESM-2 for proteins — and contributes the alignment layer binding them.

#Key Features

• Four-modality joint space: Molecule structure, free-text description, taxonomic annotation, and protein sequence are projected into one 512-dimensional space that a downstream model can consume as a single representation of a drug-target pair.
• Gramian volume alignment: The contrastive loss minimizes the volume of the parallelotope spanned by the four modality vectors, enforcing simultaneous higher-order agreement instead of a sum of independent pairwise terms.
• Adaptive modality dropout: Dropout rates follow each modality's gradient informativeness during training rather than being fixed in advance, which ablations show matters most in cold-start settings.
• IC50 weak supervision: An auxiliary regression head over the concatenated modality embeddings grounds the representation in quantitative potency where such measurements exist.
• Frozen-backbone transfer: Pretrained encoders stay frozen and only lightweight projection networks are trained, so downstream use needs just a shallow classifier head on the resulting embeddings.

#Technical Details

Pretraining uses 50,968 complete quadruplets assembled from the TRIDENT dataset extended with protein binding information from BindingDB, of which 16,035 carry quantitative IC50 values. Each modality is encoded by a frozen pretrained model — MoLFormer-XL for SMILES (768-dimensional), MolT5-base for both text descriptions and taxonomic annotations (768-dimensional), and ESM2_t33_650M for protein sequences (1280-dimensional) — and mapped by trainable projections into a shared 512-dimensional space. Downstream, the encoders remain frozen and a two-layer MLP with dropout (1024 → 512 → 256 → 2) is trained over the concatenated drug and protein embeddings.

Evaluation spans four public datasets under warm-start, drug cold-start, and target cold-start splits: Yamanishi_08 (791 drugs, 989 targets, 5,127 interactions) and Hetionet (1,384 drugs, 5,763 targets, 49,942 interactions) for DTI, plus two mechanism-of-action tasks, activation (1,426 drugs, 281 targets) and inhibition (14,049 drugs, 1,088 targets). Against baselines including CPL-GNN, MPNN-CNN, TransformerCPI, KGE-NFM, and DTIAM, GRAM-DTI reaches 0.977 AUROC on warm-start Yamanishi_08 (DTIAM: 0.967) and 0.981 on warm-start Hetionet (DTIAM: 0.957), with the clearest margin under target cold start — 0.955 versus 0.941 on Yamanishi_08. Gains are not uniform: DTIAM retains a higher warm-start AUPRC on Hetionet (0.879 versus 0.859).

#Applications

The framework targets computational chemistry and target-validation teams that already hold richer annotation than a bare SMILES-plus-sequence table — textual mechanism notes, pharmacological class labels, and measured IC50 values — and want that context to inform screening. Because encoders are frozen and only projections are learned, embeddings can be precomputed once for a compound library and target panel and reused across tasks, and the same representation supports both interaction detection and activation-versus-inhibition mechanism calls. The target cold-start results are the most operationally relevant, since prioritizing compounds against a protein with no prior assay data is the common setting.

#Impact

GRAM-DTI is a workshop paper, and its evaluation is entirely retrospective on curated public benchmarks with no prospective experimental validation. Its practical reach is limited by release status: the public repository contains pretraining code only, with no released checkpoints, no downstream evaluation scripts, and no LICENSE file despite an MIT badge in the README, so reproducing the reported numbers means reassembling the corpus and rerunning pretraining. The authors identify corpus scale as the main constraint on generalization to novel targets — requiring complete four-modality quadruplets caps pretraining at roughly 51,000 records, far smaller than the spaces the frozen backbones were themselves trained on. The lasting contribution is methodological: evidence that volume-based higher-order alignment and gradient-driven modality weighting extract more from sparse multimodal data than pairwise contrastive training does.