EnsembleEGNN

Structure-based property prediction almost always collapses a molecule to one representative conformation, even though most molecules exist in solution as a thermally populated ensemble of shapes. For cyclic peptides that simplification is costly: macrocycles interconvert between conformers with very different solvent-exposed polar surface, and passive membrane permeability depends on which shapes the molecule actually populates rather than on any single minimum-energy structure.

EnsembleEGNN, developed by the Molecular AI group at Novo Nordisk and presented at the ICML 2026 Graph Foundation Models workshop, treats the ensemble itself as the modeling unit. Each conformer is encoded independently by shared Equivariant Graph Neural Network (EGNN) layers, and the per-conformer representations are pooled by a Set Attention Block into a single, thermodynamically informed embedding. Because pooling operates over an unordered set with Boltzmann-derived weighting, the representation reflects the population of the ensemble rather than an arbitrary choice of one member.

That makes it a distinct approach among cataloged cyclic-peptide models. PeptideCLM-2 learns from SMILES strings, HELM-BERT from HELM polymer notation, and PepForge generates sequences in that notation — all working from one-dimensional chemical text. EnsembleEGNN instead learns from all-atom 3D coordinates across many conformers, and is benchmarked head-to-head against PeptideCLM-2 on the same permeability task.

#Key Features

• Ensemble as input: A molecule is a set of conformers rather than one structure, making conformational flexibility part of the representation instead of a source of noise.
• Set Attention Block pooling: Conformer embeddings are fused by two Set Attention Blocks with inducing points and a conformer-prior bias, a permutation-invariant summary weighted toward the states the molecule occupies.
• Multi-task self-supervised pretraining: Three objectives train jointly — masked token recovery over 15% of positions, denoising of Gaussian-corrupted coordinates, and reconstruction of pairwise distances against the Boltzmann-weighted mean structure.
• Pretraining as a prerequisite, not a boost: Trained from scratch on the downstream task the architecture does not learn at all; transfer from the pretraining corpus is what makes it usable.
• Parameter-efficient: At 11.3M parameters it outperforms a 114M-parameter sequence encoder on the same benchmark.

#Technical Details

The EGNN backbone uses 6 equivariant message-passing layers with a hidden dimension of 448, a 12-nearest-neighbor graph, and 0.1 dropout (7.85M parameters); conformer fusion adds two Set Attention Blocks with 4 inducing points and 8 attention heads (3.42M parameters), for 11.3M total. Pretraining uses CREMP, a dataset of roughly 36,000 macrocyclic peptides with up to 31.3M computed conformers, retaining the top five states per molecule under renormalized Boltzmann weights, for 80 epochs at batch size 8. Downstream evaluation is PAMPA permeability regression on CREMP-CycPeptMPDB (n = 2,979 after excluding assay boundary values) under 5-fold cross-validation. A randomly initialized model fails outright at R² = 0.005 ± 0.004; transferred to this different dataset, the CREMP-pretrained model reaches R² = 0.477 ± 0.009 and Pearson r = 0.699, ahead of the PeptideCLM-2 sequence-only baseline at R² = 0.439 ± 0.053 and r = 0.667. Co-training the geometric encoder end-to-end with that sequence encoder is strongest, at R² = 0.538 ± 0.029 and r = 0.737.

#Applications

The immediate use case is developability triage in cyclic-peptide drug discovery, where permeability is a common bottleneck and experimental PAMPA measurement is slow relative to the size of design libraries. Teams that already generate conformer ensembles with force fields or metadynamics can feed those ensembles straight to the encoder instead of picking a representative structure, and the hybrid configuration lets a geometric model be combined with an existing sequence encoder rather than replacing it. The conformer-pooling pattern generalizes to other properties where the ensemble matters more than the minimum, such as aggregation propensity or binding of flexible ligands.

#Impact

The paper's most transferable contribution is the demonstration that ensemble-geometry pretraining is load-bearing rather than incremental: an architecture that learns nothing from 2,979 labeled examples becomes competitive once pretrained on a large unlabeled conformer corpus, and then beats a sequence model ten times its size. That argues for geometric foundation models in a therapeutic class usually handled by chemical language models. The limits are equally clear. Evaluation covers a single property on one small dataset, so generalization across peptide property tasks is not yet established; pretraining relies on computed rather than experimental conformers; and the work is a workshop paper and arXiv preprint rather than a journal publication. The paper states that the architecture, a pretrained checkpoint, and the training code are released under an MIT license, but the repository carries no license file and no checkpoint, and there is no model-hub distribution or model card, so users pretrain and reproduce the pipeline from source. The underlying CREMP and CREMP-CycPeptMPDB corpora are separately available under CC BY 4.0.