BART-style chemical language model pretrained on 1 billion SELFIES molecules, giving property embeddings and generating valid new molecules.
No providers recorded yet. Browse all providers
A SMILES string is a program for drawing a molecule, and most random edits to it produce a program that does not run: an unclosed ring label, a stray parenthesis or an atom with too many bonds yields a string that is grammatical to a tokenizer but corresponds to no molecule at all. A chemical language model trained on SMILES has to spend capacity learning that grammar, and a decoder sampling from it can emit invalid structures. SELFIES (SELF-referencing Embedded Strings) closes that gap by construction: its tokens are defined so that every sequence of them decodes to a valid molecular graph.
SELFIES-TED pairs that representation with an encoder-decoder transformer. Developed at IBM Research in Tokyo and Brazil, it follows the BART recipe: a bidirectional encoder reads a SELFIES string in which 15% of the tokens are masked, and an autoregressive decoder rebuilds the original string token by token. The encoder's pooled hidden states serve as a molecular embedding for property prediction, while the decoder can generate molecules directly. Earlier SELFIES models such as SELFormer were encoder-only and could score molecules but not write them; SELFIES-TED can do both, and anything its decoder emits is a valid structure.
The model was first described as SELF-BART at the NeurIPS 2024 AI4Mat workshop (arXiv, October 2024), and in a longer paper under the SELFIES-TED name submitted to ICLR 2025, which adds a small variant, latent-space analysis and property-conditioned generation. It is the SELFIES component of IBM's FM4M (Foundation Models for Materials) suite, alongside the SMILES model SMI-TED and the molecular-graph model MHG-GED.
[C], [=O] or [Branch1], is a single word-level token, so tokens correspond to atoms, bonds and branch points.The large model is a BART-large configuration: 12 encoder and 12 decoder layers, hidden size 1,024, 16 attention heads, feed-forward size 4,096, and a 3,160-token SELFIES vocabulary, for roughly 358 million parameters. It was pretrained on about 1 billion molecules drawn from ZINC-22 and PubChem, converted from SMILES to SELFIES with the SELFIES library, using a denoising objective. The small variant has 2 encoder-decoder layers, 4 heads and hidden size 256, with a 173-token vocabulary, and was pretrained on 8 billion ZINC-22 samples. Training used NVIDIA V100 16 GB GPUs.
In the SELF-BART evaluation, frozen embeddings were fed to an XGBoost model tuned with Optuna on nine MoleculeNet datasets using the standard splits. ROC-AUC reached 95.2 on BBBP, 96.9 on ClinTox, 83.0 on HIV and 89.3 on BACE, the best in the comparison on three of the four and tied with N-Gram on HIV, ahead of MoLFormer-XL throughout; it trailed on SIDER (65.0) and Tox21 (76.5). Regression RMSE was 0.454 on ESOL and 1.397 on FreeSolv, the best in the comparison, and 0.771 on Lipophilicity, behind several graph models. For unconditioned generation, 10,000 molecules sampled from the decoder showed validity 0.998, uniqueness 0.999, novelty 1.0 and internal diversity 0.918, though the baselines were trained on MOSES rather than the same corpus.
SELFIES-TED is a featurizer for medicinal and materials chemists with small labeled datasets: encode a compound library once, then fit a light model for blood-brain barrier permeability, clinical toxicity, solubility or hydration free energy. Because the decoder generates valid structures, it also supports exploratory molecule generation and latent-space manipulation for design. Through FM4M-Kit and the hosted FM4M web demo, users can combine its embeddings with SMILES- and graph-based models without writing model code.
SELFIES-TED puts the validity guarantee of SELFIES into a generative encoder-decoder model with released weights, and it anchors the SELFIES modality in IBM's open FM4M suite, which the lab has promoted through tutorials and the AI Alliance. Both checkpoints and the code are published under Apache 2.0. The evidence base is thin: the arXiv paper is a four-page workshop contribution, the fuller SELFIES-TED paper appears only as an ICLR 2025 submission on OpenReview rather than a peer-reviewed publication, the benchmarks are in silico, and the curated 1-billion-molecule pretraining set is not distributed, although its ZINC-22 and PubChem sources are public.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.