Chem42

Most chemical language models learn the grammar of molecules in isolation. They can complete a SMILES string, estimate solubility, or sample a plausible new structure, but they have no notion of the protein the molecule is supposed to bind. That gap forces a two-stage workflow — generate broadly, then filter by docking — which wastes most of the generated candidates. Chem42 is built to close it, generating ligands that are conditioned on a specific biological target from the start.

Chem42 comes from the Inception Institute of Artificial Intelligence in Abu Dhabi with Cerebras Systems, and is one component of the group's Omics42 platform alongside Gene42, a genomic language model, and Prot42, a protein language model. The three are designed to interoperate, and Chem42's target-awareness is a direct consequence: protein sequence embeddings produced by Prot42 are injected into Chem42's decoder through cross-attention, so the molecular representation the model builds is shaped by the binding pocket it is designing against.

The family spans four sizes and is evaluated across three task types — molecular property prediction, property-conditioned molecule generation, and target-aware ligand design — with the same pretrained checkpoints applied zero-shot across diverse protein targets. The work is a preprint and has not been peer reviewed.

#Key Features

• Cross-modal target conditioning: Protein embeddings serve as keys and values while ligand tokens act as queries in cross-attention layers placed before the decoder blocks, with a residual connection back to the original molecular representation.
• Atom-level tokenization: A 268-token vocabulary covering periodic-table elements, SMILES syntax, and control symbols encodes molecules atom by atom rather than through subword merges, preserving element identity in the sequence.
• Randomized SMILES augmentation: Pretraining corpora are expanded with multiple random SMILES enumerations of each molecule, teaching the model that a structure has many valid string forms.
• A size ladder for different budgets: Four checkpoints from 190M to 1B parameters let property prediction, conditional generation, and ligand design each run at the scale that task needs.

#Technical Details

Chem42 is a decoder-only autoregressive transformer in the LLaMA mold, using rotary positional embeddings and a 512-token context window. The 1B configuration has 24 layers, a hidden dimension of 1,792, 32 attention heads, and 8 key-value heads; smaller variants come in at 190M, 387M, and 597M parameters. Pretraining draws on UniChem, with four corpus variants: 188.4M canonical molecules (9.5B tokens) at the base, scaling through random-enumeration augmentation to 942.3M molecules and 49.5B tokens. The authors target a tokens-per-parameter ratio of 50. Training ran on Cerebras CS-2 wafer-scale systems.

On MoleculeNet property prediction, Chem42-1B improves on ChemFM-3B in 10 of 12 tasks, including a ClinTox ROC-AUC of 0.971 against 0.899 and a PDBBind full-set RMSE of 0.674 against 0.700. For property-conditioned generation on GuacaMol-style benchmarks, Chem42-597M reaches 0.996 validity when conditioning on synthetic accessibility and a mean absolute deviation of 0.043 when conditioning on QED, ahead of both MolGPT and ChemFM-3B. Target-aware ligand design was assessed across nine protein targets, where generated ligands achieved QED between 0.71 and 0.90 with synthetic accessibility scores exceeding a pocket-conditioned baseline. All reported evaluation is computational; no experimental validation of the designed ligands is presented.

#Applications

The intended use is early-stage drug discovery, specifically the transition from a protein target to a shortlist of candidate ligands. A team with a sequence for a target of interest can generate structures already biased toward that pocket, filtered for chemical validity and estimated synthesizability, rather than sampling a general chemical library and discarding most of it downstream. The same checkpoints support property prediction on existing compounds and property-conditioned generation when the objective is a desired QED or synthetic accessibility profile instead of a specific target. Because targets enter as sequence embeddings, no experimental structure is required.

#Impact

Chem42's contribution is architectural: it shows a concrete mechanism for making a chemical language model target-aware by borrowing representations from a protein model in the same family, rather than bolting a docking score onto a sampling loop. Its practical reach is limited by availability. The preprint states that the models are available at huggingface.co/inceptionai, but that organization hosts only the group's Jais general-purpose language models; no Chem42 checkpoint, code repository, hosted API, or license terms are public, and neither is sibling Prot42, on which the target-aware pathway depends. Until those artifacts appear, the reported benchmarks cannot be independently reproduced, and the model is best read as a design and a set of results rather than a tool a laboratory can pick up.