PG-MLD

Molecular language models trained on SMILES strings scale cheaply and infer fast, but a one-dimensional string says nothing explicit about where atoms sit in space or how a molecule flexes over time. Three-dimensional models capture that geometry, but require conformer generation and, for dynamics, simulation — costs that are prohibitive when screening large libraries. PG-MLD (Physics-Guided Molecular Representation Learning via Dynamic 3D Trajectory Distillation) resolves the tradeoff by paying the 3D cost once, during pretraining, and then discarding it.

The framework trains a "dynamic 3D physical teacher" on molecular dynamics trajectories and distills what it learns into SMILES-based student language models. The teacher pairs an equivariant geometric encoder, applied frame by frame, with a Liquid Time-Constant network that models conformational evolution across frames in continuous time. Its atom-level and molecule-level representations are then transferred to a student through representation alignment and cross-modal contrastive learning, with masked language modeling retained for students that support it. After distillation the teacher is thrown away: downstream prediction runs on SMILES alone, with no conformer generation and no simulation.

PG-MLD was developed by researchers at Qingdao University of Science and Technology, with collaborators at KAUST and the University of Science and Technology of China, and posted to bioRxiv in August 2026. It departs from earlier cross-modal distillation work, which derives geometric supervision from a single conformer or a small static ensemble, by treating the trajectory itself — atomic motion, conformational transitions, and their time ordering — as the signal.

#Key Features

• Trajectory-level physical supervision: The teacher's input view combines an initial conformer with coordinate Gaussian noise, conformational angle noise, and temporally ordered molecular dynamics frames, giving it dynamics rather than a frozen snapshot.
• Liquid Time-Constant temporal encoder: Learnable time constants adaptively regulate the state-update rate across frames, so slow scaffold motion and fast local fluctuation are modeled at their own time scales instead of a uniform discrete step.
• Atom-level electronic descriptors: Formal charges and Gasteiger partial charges are carried through the teacher, adding local physicochemical environment on top of geometry.
• Two-level alignment: A token–atom mapping lets the student's SMILES tokens be aggregated per atom, enabling atom-level and molecule-level alignment with the frozen teacher in shared projected latent spaces.
• SMILES-only inference: The distilled student needs nothing but a SMILES string at prediction time, preserving the throughput advantage of sequence models.
• Architecture-agnostic transfer: The same recipe was applied to three different student language models without architectural changes.

#Technical Details

The distillation objective is a weighted sum of four terms: mean-squared alignment losses at the atom and molecule levels, a cross-modal contrastive loss that pulls the 3D and 1D views of the same molecule together while separating different molecules, and a masked language modeling loss whose weight is set to zero for students without an MLM head. Separate projection heads reconcile the differing dimensionality and feature distributions of teacher and student. After distillation the student encoder is paired with a task head and fine-tuned per task.

Evaluation covers nine MoleculeNet datasets under matched random splits, reporting mean and standard deviation over five seeds. PG-MLD reaches ROC-AUC of 0.966 (BBBP), 0.936 (BACE), 0.997 (ClinTox), 0.851 (Tox21), 0.744 (ToxCast), and 0.825 (HIV), and RMSE of 0.627 (ESOL), 0.945 (FreeSolv), and 0.620 (Lipophilicity), against baselines including GEM, KANO, KPGT, and GraphMVP. Applied to ChemBERTa, ChemBERTa-2, and MoLFormer as students, distillation improves 25 of 27 model–task settings — ChemBERTa's BACE ROC-AUC rises from 0.901 to 0.927, MoLFormer's Lipophilicity RMSE falls from 0.662 to 0.620 — with a small regression on ChemBERTa/Tox21 and ChemBERTa-2/Lipophilicity. Distilled embeddings also retrieve more chemically sensible neighbors (mean ECFP Tanimoto similarity of the top-3 rising from 0.393 to 0.631) and separate classes better (Davies–Bouldin index dropping from 0.95 to 0.85 on BACE).

#Applications

The practical target is high-throughput property prediction in drug discovery: ADMET and toxicity screening, aqueous solubility, and lipophilicity estimation over libraries where generating conformers for every candidate is not viable. Because the output is a distilled checkpoint of a standard molecular language model, cheminformatics teams already running ChemBERTa- or MoLFormer-style pipelines can substitute a physics-informed encoder without changing their input format or inference budget.

#Impact

PG-MLD makes a specific methodological argument — that dynamic, trajectory-derived physics is a transferable training signal for sequence models, and that its benefit survives the move to a different student architecture. That claim rests on a bioRxiv preprint that has not been peer reviewed. The code is public, but the repository carries no license file, and its README states that base language-model weights and PG-MLD's own pretraining and downstream checkpoints are not stored there and must be downloaded or generated separately; the preprint is posted under bioRxiv's all-rights-reserved terms, which permit no reuse without permission. Reproducing the reported numbers therefore requires supplying molecular dynamics trajectories and rerunning distillation.