CL-MFAP

Antimicrobial resistance keeps outpacing the antibiotic pipeline, and the screening step — deciding which of millions of candidate compounds is worth synthesizing and assaying — is where machine learning has the most leverage. Most chemical foundation models attack this with a single view of a molecule, usually a SMILES string, trained over enormous general-purpose compound libraries. CL-MFAP, from Western University and the University of Toronto, argues that both choices are suboptimal for antibiotic discovery: a molecule has several complementary representations, and a smaller corpus of bioactive, drug-like chemistry teaches more about the properties that matter than a much larger corpus of arbitrary structures.

The model therefore pretrains three encoders jointly, one per modality — a linearized SMILES string, a molecular graph, and a Morgan fingerprint — and aligns their embedding spaces with a contrastive objective across all three pairings. No labels are required. After pretraining, the checkpoint is fine-tuned for downstream property prediction, with antibiotic activity as the target application.

Published at ICLR 2025, CL-MFAP sits in the same family as chemical language models such as ChemBERTa and MoLFormer, but differs in being explicitly multimodal at pretraining time and domain-focused in its corpus.

#Key Features

• Three complementary molecular views: SMILES text, molecular graph topology, and Morgan fingerprints are encoded separately and forced into a shared space, so each modality's blind spots are covered by the others.
• Cross-modal contrastive objective: All three pairings — SMILES/fingerprint, SMILES/graph, and fingerprint/graph — are trained with an NT-Xent loss, giving three alignment signals from unlabeled molecules.
• Bi-level routing attention on graphs: The graph encoder first scores coarse regions for relevance and then attends finely only within the top-ranked ones, concentrating capacity on the substructures that drive activity.
• Rotary embeddings for SMILES: The sequence encoder uses rotary position embeddings, which help relate distant tokens in a linearized structure where bonded atoms can be far apart in the string.
• Domain-focused pretraining corpus: Training on bioactive, drug-like molecules rather than general chemistry targets the region of chemical space where antibiotic candidates live.

#Technical Details

CL-MFAP is pretrained on 1.6 million bioactive drug-like molecules drawn from ChEMBL after salt removal and preprocessing. The SMILES branch is a 12-layer transformer encoder with rotary position embeddings; the graph branch is a 12-layer Graphormer-style transformer augmented with bi-level routing attention, optionally combined with a message-passing network; the fingerprint branch is a multilayer perceptron over Morgan fingerprints. The three are trained jointly with pairwise NT-Xent contrastive losses. Downstream evaluation fine-tunes the pretrained checkpoint on six datasets: minimum inhibitory concentration against E. coli (curated from COADD) and against H. influenzae (from ChEMBL), plus BACE, blood-brain barrier penetration, PAMPA permeability, and bioavailability, each split 80/10/10. On the E. coli MIC task CL-MFAP achieves the best ROC-AUC among all baselines at 0.854 ± 0.037, and on H. influenzae MIC it reaches 0.874 ± 0.015, statistically indistinguishable from the best-performing MoLFormer at 0.876 ± 0.017. Across the six tasks it ranks ahead of MolBERT, MolCLR, and FP-GNN. The public repository contains pretraining and fine-tuning code with a checkpoint-loading workflow, but carries no license file and does not distribute pretrained weights.

#Applications

The intended workflow is virtual screening for antibacterial activity: embed a candidate library with the pretrained encoders, fine-tune on whatever labeled MIC or activity data a group has, and rank compounds for synthesis and assay. Because the pretraining is label-free and modality-agnostic, the same checkpoint transfers to general ADMET-style endpoints — permeability, bioavailability, blood-brain barrier penetration — which matter for turning an active compound into a viable drug. Medicinal chemistry and antimicrobial groups with modest labeled datasets are the natural users, since the contrastive pretraining is what supplies the data efficiency.

#Impact

CL-MFAP's contribution is methodological as much as practical: it shows that a domain-scoped corpus of 1.6 million bioactive molecules, exploited across three modalities, can match or beat chemical language models pretrained on tens of millions of general compounds on the tasks that matter for antibiotic discovery. That is a useful counterweight to the assumption that corpus size dominates in molecular representation learning. Adoption remains early — the repository has limited traction, no weights are distributed, and no license has been declared — so the model is currently more valuable as a reproducible recipe for multimodal chemical pretraining than as an off-the-shelf screening tool.