ShortListing Model (SLM)

The ShortListing Model (SLM) is a discrete generative architecture built for sequences that have no natural left-to-right ordering, the situation that describes most of biology: an enhancer, a promoter, or a protein is read as a whole rather than composed one token at a time. It was developed by the Generative Symbolic Intelligence lab and the Institute for AI Industry Research at Tsinghua University, with co-authors at ByteDance Seed, and presented at NeurIPS 2025.

SLM belongs to the simplex-based family of discrete diffusion models, which represent a categorical variable as a point on the probability simplex so that the generative trajectory respects the geometry of a distribution over letters. Earlier members of that family pay a heavy price for this: statistical and Fisher flow matching define geodesics under a Fisher-Rao metric with Riemannian optimal transport, and Bayesian Flow Networks require change-of-variable machinery over Gaussian count variables. SLM replaces the continuous trajectory over the whole simplex with progressive candidate pruning. Generation starts with every category in play and iteratively shortlists, jumping between the centroids of simplex subfaces until one category remains. This cuts the degrees of freedom the model must track and, combined with a cross-entropy objective instead of the vocabulary-level squared-error losses used by its predecessors, avoids the vanishing gradients that hobble simplex methods at large vocabularies.

The architecture is domain-agnostic and the paper benchmarks it on language modeling as well, but the released biological checkpoints are what make it usable in sequence design: separate models for cell-type-conditioned DNA enhancer generation and for unconditional protein generation. The protein checkpoint follows the training protocol of EvoDiff, the established discrete-diffusion baseline for sequence-space protein design.

#Key Features

• Shortlisting formulation: Generation is modeled as progressive elimination of candidate categories, with transitions restricted to simplex centroids rather than the full continuous simplex, reducing the degrees of freedom to be learned.
• Cross-entropy objective: A simplified cross-entropy loss replaces the vocabulary-level MSE losses of earlier simplex methods, mitigating vanishing gradients and scaling to large vocabularies.
• Flexible classifier-free guidance: Conditional generation is steered by a guidance factor that trades fidelity against diversity, and the model is comparatively insensitive to how that factor is tuned.
• Released biological checkpoints: Fixed weights are published for fly-brain and human-melanoma enhancer generation and for UniRef50-trained protein generation, loadable for sampling without retraining.
• One architecture across substrates: The same formulation trains on DNA, protein, and text, so improvements transfer between sequence domains rather than being tied to a single alphabet.

#Technical Details

The protein model has 38 million parameters, matching the small EvoDiff configuration, and was trained on the 42 million sequences of UniRef50 for 200,000 steps with a maximum length of 1024. Evaluated on unconditional generation, it exceeds all diffusion and flow-matching baselines, including both EvoDiff variants, MDLM, and discrete flow matching, on foldability measured by ESMFold pLDDT, fitness measured by ProGen2-xlarge log-likelihood, self-consistency measured by ESM-IF perplexity, and structural diversity measured by pairwise TM-score, and is competitive with the 150-million-parameter ESM2 masked language model.

The DNA models are trained per task. Enhancer checkpoints cover 104,000 fly-brain sequences across 81 cell-type classes and 89,000 human melanoma sequences across 47 classes, each 500 base pairs, with labels from ATAC-seq. Without guidance, SLM reaches a Frechet Biological Distance of 2.2 on melanoma and 4.4 on fly brain against 3.3 and 10.8 for a Bayesian Flow Network and 5.3 and 15.1 for Dirichlet flow matching; with classifier-free guidance these fall to 1.4 and 1.0. On conditional promoter design over 100,000 human promoter sequences of 1024 base pairs, SLM attains a mean squared error of 0.0265 against the target regulatory profile, the lowest among autoregressive, bit-diffusion, discrete-diffusion, and simplex baselines.

#Applications

The enhancer checkpoints support cell-type-specific regulatory element design, the task underlying synthetic enhancers for gene therapy vectors and targeted expression constructs, where the aim is activity in one cell population and silence elsewhere. The protein checkpoint generates foldable, natural-looking sequences as a starting pool for design campaigns. More broadly, the architecture is a drop-in generative backbone for sequence design problems with a modest alphabet and no inherent ordering.

#Impact

SLM's contribution is to make simplex-based discrete diffusion practical. Where earlier simplex methods traded tractability for mathematical elegance and often failed outright at large vocabularies, SLM keeps the gradual information growth that motivates the approach while reducing it to transitions among a finite set of centroids trained with a standard cross-entropy loss. The code carries an MIT license and the DNA and protein checkpoints are hosted on Hugging Face, though the repository provides no model card or data card documenting intended use, training composition, or limitations, and no promoter checkpoint is distributed despite promoter design being reported in the paper. The biological results reproduce established narrow benchmarks rather than demonstrating one generalist model, so SLM is best understood as a single architecture with a family of task-specific checkpoints.