SELEX MLM

Systematic Evolution of Ligands by Exponential Enrichment (SELEX) discovers aptamers by cycling a random single-stranded DNA library through rounds of binding, washing, and PCR amplification. The process is easy to run and hard to read: each round returns tens of millions of sequences, and deciding whether the library has genuinely converged on a target — rather than on the beads, the linker, or the amplification chemistry — normally means stopping to run binding assays. This model, developed by Caris Life Sciences and Arizona State University, treats the library as a text corpus instead, learning an embedding in which enrichment itself becomes a measurable quantity.

A masked language model was pretrained on 10.4 million unlabeled 35-nucleotide sequences pooled from six independent SELEX enrichments against protein and peptide targets, together with the unenriched starting libraries. Because sequences from every library were mixed without labels, the resulting checkpoint captures an overall view of enrichment rather than a target-specific one. It is then applied frozen: sequences from any pair of libraries are embedded, projected with UMAP, and compared by kernel density estimate overlap. The same frozen backbone supports supervised classifiers that assign a sequence to the library it came from.

Where genomic DNA language models such as DNABERT learn from natural genomes, this model learns from a synthetic randomized library whose only structure is the selection pressure applied to it. The authors give the model no brand name; the paper refers to it simply as the pre-trained model.

#Key Features

• Label-free training signal: Pretraining uses nucleotide sequence only — no copy numbers, no target identity, no binding measurements — yet the latent space still separates enriched from unenriched populations.
• Enrichment as a number: Latent-space overlap between an enriched library and its unenriched starting pool tracks measured target binding, with a Pearson correlation of -0.85, giving a sequencing-only readout of selection progress.
• Fine-grained target discrimination: Classifiers separate libraries enriched against 30-residue peptides that differ by a single amino acid, and separation scales with how different the two targets are.
• Replicate-stable representations: Independent enrichments against the same target, started from entirely different random libraries, land in nearly the same region of latent space.
• Weights and data released: The Zenodo deposit carries the training and holdout sequence sets, a label map, and the pretrained model itself under CC BY 4.0.

#Technical Details

The architecture is three convolutional layers feeding two stacked bidirectional LSTM layers with 256 units, concatenated into a time-distributed dense layer that emits a per-token prediction. Training masked 15% of nucleotides and recovered them, running for 500 epochs in TensorFlow and Keras. Of roughly 66.5 million unique sequences across all libraries, 500,000 were sampled per library into a 10.4 million-sequence training set (fewer in three low-depth cases), leaving more than 53 million sequences as a holdout the model never saw.

For classification the pretrained weights were fixed and two dense layers added, trained on an 80:20 split across all 190 unique pairs among 20 sequence sets and evaluated on holdout sequences. Pairs of unenriched libraries gave AUC 0.50, as did in-silico random controls; enriched-versus-enriched comparisons ranged from 0.52 to 0.84. Replicate enrichments sat near 0.50, single-amino-acid target differences moved it to about 0.67, and targets overlapping by only 11-12 of 30 residues reached 0.76-0.79. Classifier AUC and kernel density overlap were anticorrelated at -0.97, and permutation tests with 10,000 label shuffles cleared p < 0.05 for every comparison except unenriched versus unenriched.

#Applications

The immediate use is process control for aptamer discovery: a group running SELEX can sequence a round, embed it, and quantify how far it has moved from the starting library without pausing for a binding assay. The same measurement flags rounds that enriched on the bead or linker rather than the intended target, and confirms that replicate selections converged. Because the classifier resolves targets differing by one residue, the approach also supports specificity engineering — checking whether a library discriminates a mutant protein domain from its wild-type counterpart, as demonstrated here on the Von Willebrand Factor A1 domain and its 2B variant.

#Impact

The work establishes that target-specific information is recoverable from enriched sequence content alone, without labels or structural modeling, and that a single frozen checkpoint transfers across library pairs it was never trained to compare. Its scope is narrow by construction: every target belongs to one closely related family, all sequences share a 35-nucleotide format, and evaluation is confined to that setting, so generalization to unrelated target classes is untested. No code repository accompanies the release, and the preprint has not been peer reviewed. For groups already generating deep sequencing across SELEX rounds, the released checkpoint nonetheless offers a concrete way to turn that data into an enrichment metric.