Chem-xLSTM

A generative model trained on ChEMBL will happily produce plausible drug-like molecules, but a project rarely wants drug-like in general — it wants another PROTAC, another macrocyclic natural product, another member of a DNA-encoded library series. The usual answer is to fine-tune the generator on whatever examples of that series exist, which means a training run and a fresh checkpoint for every domain a chemist cares about, each one spent on very little data.

Chem-xLSTM moves that conditioning out of the weights and into the prompt. Paste six SMILES strings from the series you want into the context, and the model continues the sequence with new molecules from the same region of chemical space — no gradient steps, no per-domain checkpoint. It is the chemistry member of Bio-xLSTM, a suite of xLSTM architectural variants from the ELLIS Unit Linz and LIT AI Lab at Johannes Kepler University Linz with NXAI, published at ICLR 2025. The same report introduces Prot-xLSTM for proteins and DNA-xLSTM for genomes; each sibling has its own repository, corpus and checkpoints.

The backbone is what makes long conditioning contexts affordable. xLSTM revives the LSTM with exponential gating and a matrix memory cell; its mLSTM block parallelises across the sequence during training the way attention does, but at inference it decodes recurrently in constant memory with runtime linear in sequence length, rather than carrying a key–value cache that grows with every token.

#Key Features

• In-context domain conditioning: A prompt of example SMILES from one molecular domain steers generation into that domain, so adapting the model to a new chemical series requires no fine-tuning and no new weights.
• Matrix-memory recurrent blocks: mLSTM layers replace self-attention with an exponentially gated matrix cell state, giving linear-time training and constant-memory decoding.
• Two released checkpoints for two jobs: an unconditional generator trained on ChEMBL at a 100-token context, and a domain-conditioned model trained at a 4,096-token context.
• A conditioning corpus built for the task: the In-Context Style Transfer set concatenates molecules from 249 curated domains — natural products, click chemistry, PROTACs, DNA-encoded libraries, drugs, bioassay actives — and holds out whole domains for validation and test.
• Distribution-faithful sampling: on unconditional generation it matched the reference chemical distribution more closely than any architecture compared at equal size.

#Technical Details

Both released models carry roughly 15M parameters in 9 mLSTM blocks with an embedding dimension of 512 and a 1.3:1 projection ratio. The unconditional model, Chem-xLSTM-15M, was trained by causal language modelling on 1.9 million canonical SMILES from ChEMBL v31 at a 100-token context. Against LSTM, GPT, S4 and Mamba baselines held at the same parameter count and evaluated on 102,400 generated strings, it reached the lowest Fréchet ChemNet Distance at 0.13 — ahead of SMILES-GPT at 0.15, Chem-Mamba at 0.21, SMILES-S4 at 0.28 and SMILES-LSTM at 0.46 — with a competitive perplexity of 1.68 against SMILES-GPT's best 1.65; every architecture generated valid, unique and novel molecules. The conditional model, Chem-xLSTM-15M-icl, was trained on the In-Context Style Transfer set of more than four million molecules across 249 domains, split 8:1:1 by domain so that validation and test domains are entirely unseen, with permutation augmentation over molecule order and a 4,096-token context; on next-token loss over those unseen domains it beat the Mamba, Transformer++ and S4 models trained the same way. Both checkpoints load through xLSTMforNTP.from_file and generate directly, as a Colab notebook in the repository demonstrates for 6-shot conditional design.

#Applications

The intended use is de novo design inside a defined chemical series: assemble whatever known members exist — a natural-product collection, a set of covalent binders, actives from one bioassay — put them in the context, and sample analogues from the same distribution for triage. Because conditioning is a prompt rather than a training run, a medicinal chemistry team can iterate through many series on a single GPU and steer a run by editing the examples. The unconditional checkpoint serves the separate job of generating broad drug-like libraries or scoring SMILES by likelihood.

#Impact

Chem-xLSTM's contribution is architectural evidence: a recurrent model with matrix memory matches or beats state space models and transformers at chemical language modelling, and its constant-memory decoding pays off exactly where in-context conditioning wants long prompts. The models are deliberately small — 15M parameters, chosen to match the baselines rather than to push scale — and the authors flag that behaviour beyond the billion-parameter regime is untested and that the training sets carry the exploration biases of the databases they came from. The code is Apache-2.0 licensed, but the checkpoints are distributed as a single archive on institutional cloud storage rather than a model hub and carry no licence of their own. Sampling is not guaranteed to yield parseable structures: the repository's own few-shot notebook filters generated strings through RDKit before displaying them.