Multimodal molecular model that fuses 2D molecular graphs, biomedical text and knowledge-graph neighbours for retrieval, captioning and generation.
No providers recorded yet. Browse all providers
A drug's identity is spread across three kinds of record. Its structure says which atoms are bonded to which; a paragraph in the literature says it is a sulfonamide that inhibits soluble epoxide hydrolase; and a knowledge base says which proteins it binds, which diseases it treats and which other compounds resemble it. Earlier molecule-text models used at most the first two. Generative models such as MolT5 and KV-PLM read the structure only as a SMILES string, which hides ring topology, while contrastive models such as MoMu align a whole-molecule embedding with a whole-paragraph embedding and never learn which substructure a phrase refers to. None of them used the knowledge graph.
MolFM, from the Institute for AI Industry Research (AIR) at Tsinghua University, treats all three as input modalities of one model. A multimodal transformer lets each word of a description attend jointly to the atoms of the molecular graph and to the molecule's neighbours in a biomedical knowledge graph, so a phrase like "contains a trifluoromethyl group" can be grounded on the atoms that form it, and "is used to treat pancreatitis" on the disease node. The authors frame the pretraining objectives as metric learning: they pull the three views of the same molecule together, and through the knowledge-graph term they also pull together molecules that share structure or act on the same target or disease.
The model was posted to arXiv in June 2023 and released inside the OpenBioMed toolkit, now maintained by PharMolix alongside BioMedGPT-10B and PharMolixFM. It is unrelated to UBio-MolFM, a machine-learning force field that shares the name.
MolFM has four components, about 138M parameters in total: a 5-layer GIN structure encoder initialized from GraphMVP (1.8M parameters), a 6-layer text transformer initialized from the first six layers of KV-PLM (61.8M), a TransE knowledge-graph encoder (12.6M) pretrained for 500 epochs on the graph, and a 6-layer multimodal encoder with cross-attention initialized from the last six layers of KV-PLM (61.8M). Pretraining combines four objectives: structure-text contrastive loss, cross-modal matching (is this structure-text-knowledge triple from one molecule?), masked language modelling conditioned on all three modalities, and the knowledge-graph embedding loss. It follows MoMu's pretraining data, 15K PubChem molecules paired with 37M paragraphs from S2ORC, and adds a knowledge graph of 49K entities and 3.2M relations built from public databases. Training ran for 300 epochs at batch size 128 on four NVIDIA A100 GPUs.
On paragraph-level retrieval on PCdes, zero-shot MolFM reaches an MRR of 21.42 (structure to text) and 23.63 (text to structure), against 9.89 and 10.33 for MoMu; fine-tuned, it reaches 39.56 and 39.34. With a MolT5-base decoder on ChEBI-20 it scores BLEU-4 0.498 for captioning and an exact-match rate of 0.210 for text-to-molecule generation. On eight MoleculeNet tasks it averages 74.62 ROC-AUC with text and knowledge inputs, and 73.95 from structure alone. In ablations, removing the knowledge graph cut zero-shot retrieval recall by about 1.5 points on average, and removing cross-attention to atoms cut it by about 3 points.
MolFM suits medicinal chemists and cheminformaticians who need to search between structures and prose: finding the literature description that matches a compound, retrieving candidate structures for a textual specification, or drafting a description for an uncharacterized molecule. Its structure encoder can also be fine-tuned for property prediction such as toxicity or BACE inhibition. The authors state that it should be used for research only, and note that it helps less with newly reported molecules, which have little text or knowledge-graph context.
MolFM showed that a biomedical knowledge graph can serve a molecule-language model as a third input modality, rather than only as a source of data augmentation, and that cross-attention can ground individual phrases on specific atoms and related entities. Its checkpoint and code were distributed through OpenBioMed and fed into the same group's later work: the DrugFM model in the same toolkit incorporates MolFM, and later multimodal molecular models such as M2UMol use it as a comparison baseline. Its benchmarks are all in silico, and its pretraining set of 15K molecules is small by later standards. The three-modality model is also easier to read about than to rerun: the released class has the knowledge-graph encoder wiring commented out, and neither the pretraining loop nor the 49K-entity graph was published.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.