GeoRecon

Self-supervised pretraining transformed language and vision by inventing tasks — masked tokens, next-token prediction — whose solutions happen to require exactly the representations downstream work needs. Molecular representation learning has borrowed the recipe but converged on a narrow pretext task: perturb atomic coordinates with noise and train the network to denoise them. That objective is well motivated, since coordinate denoising is provably equivalent to learning a force field under a Boltzmann assumption, and it teaches the encoder a great deal about local atomic environments. It teaches it comparatively little about the molecule as a whole, which is a problem when the downstream target is a graph-level quantity such as total energy or enthalpy.

GeoRecon, from the Institute for Artificial Intelligence at Peking University, reframes the pretext task at the graph level. Alongside the usual node-level denoising head, it adds a reconstruction head that must recover the molecule's geometry conditioned on a single pooled graph embedding. Because that embedding is the only channel through which global information can reach the reconstruction, the encoder is forced to compress coherent whole-molecule structure into it rather than relying on per-atom features alone. The framework requires no external labels, no 2D graph inputs, and no cheminformatics-derived conformers.

The measurable consequence is a much smoother representation. Estimating the encoder's local non-rigid Lipschitz constant — how far the representation moves when an atom is nudged — GeoRecon sits near 30, while the node-denoising Coord baseline exceeds 10,000, a reduction of roughly three orders of magnitude. The authors connect that smoothness to tighter generalization bounds and to the empirical fine-tuning gains they observe. The work first appeared in June 2025.

#Key Features

• Graph-level reconstruction pretext task: Geometry is reconstructed from a pooled molecular embedding rather than from per-atom states, so the encoder must encode emergent whole-molecule structure instead of only local neighborhoods.
• Multi-task objective with denoising retained: The node-level denoising head is kept alongside reconstruction, preserving the force-field equivalence that makes denoising effective while adding the missing graph-level signal.
• Smoother latent manifold: Pretraining cuts the encoder's local Lipschitz constant by nearly three orders of magnitude relative to coordinate denoising, making representations markedly more robust to small structural perturbations.
• Label-free and modality-clean: No external supervision, no 2D molecular graphs, and no RDKit-generated conformers are required — the method operates directly on 3D atomic coordinates.
• Sample efficiency: In a controlled ablation, pretraining on a 100k-molecule subset with a 10-layer encoder outperforms baselines pretrained on the full corpus, indicating the objective extracts more per molecule than denoising alone.

#Technical Details

The backbone is an 8-layer SE(3)-equivariant transformer encoder adapted from TorchMD-Net, operating on atomic numbers and Cartesian coordinates, with separate denoising and reconstruction decoders on top. Pretraining runs on PCQM4Mv2, and downstream evaluation spans four benchmarks. On QM9's 134k DFT-optimized small organic molecules, GeoRecon achieves the best result among all baselines on the internal energy, ground-state energy, enthalpy, and free energy targets, with improvements over its Coord backbone on several others. On MD17 force prediction it improves on Coord for every molecule tested, with relative MAE reductions of 11.37% on toluene, 10.43% on ethanol, 10.05% on salicylic acid, and 9.45% on aspirin. On the larger supramolecular systems of MD22 the energy gains are bigger still: 24.20% on DHA and 14.70% on stachyose, with smaller but consistent force improvements. An out-of-distribution evaluation on the flexible drug-like molecule 3BPA and a frozen-encoder linear-probing experiment both favor GeoRecon over the same backbone trained with denoising alone.

#Applications

The immediate users are computational chemists and machine-learned force field developers who fine-tune a pretrained 3D encoder for quantum property regression or for energy and force prediction in molecular dynamics. Improvements on MD22-scale systems — peptides, lipids, carbohydrates, and nucleobase pairs in the 42-to-370-atom range — are the most relevant for biomolecular simulation, where non-local interactions dominate and node-centric pretraining struggles. Because GeoRecon is an objective rather than an architecture, it can be layered onto other SE(3)-equivariant encoders.

#Impact

GeoRecon's argument — that molecular pretraining has over-indexed on node-level tasks, and that forcing global information through a pooled embedding both improves accuracy and provably smooths the latent space — is a clean and portable one. The evidence is honest about its scope: gains are measured principally against the authors' own Coord backbone rather than a broad sweep of state-of-the-art force fields, downstream use requires fine-tuning rather than frozen zero-shot transfer, and evaluation is confined to small organic molecules and modest supramolecular systems. Extension to biomacromolecules is named as future work. The paper is a preprint that has not been peer reviewed, and while code was supplied as supplementary material, no public repository or pretrained weights have been released.