LOL-EVE

Disease-associated genetic variation frequently falls in noncoding regulatory regions, yet interpreting these variants remains far harder than for coding mutations. Promoters are especially challenging: they harbor short insertions and deletions (indels) that reshape transcription-factor binding, and most genomic language models score only single-nucleotide substitutions and offer no principled way to evaluate length-changing edits. LOL-EVE addresses this gap for the promoter regulatory genome.

LOL-EVE is a conditional autoregressive transformer trained on 13.6 million mammalian promoter sequences. By conditioning generation on gene- and species-level context drawn from evolutionary sequence alignments, the model learns the distribution of viable promoter sequences and assigns likelihood-based scores to variants without task-specific supervision. This design enables both zero-shot indel effect prediction and complete promoter sequence scoring, extending the evolution-based variant-effect philosophy the Marks Lab pioneered for proteins (EVE) into noncoding DNA.

Developed by the Marks Lab in the Department of Systems Biology at Harvard Medical School together with the Broad Institute of MIT and Harvard, LOL-EVE was first released as a preprint in November 2024 and updated in October 2025.

#Key Features

• Conditional autoregressive modeling: Generation is conditioned on gene and species context, letting the model capture evolutionary constraints specific to each promoter rather than a single genome-wide prior.
• Zero-shot indel prediction: LOL-EVE scores insertions and deletions directly, a capability most genomic language models lack because they are restricted to fixed-length substitution scoring.
• Evolutionary training corpus: Pretraining on 13.6 million mammalian promoter sequences exposes the model to broad cross-species regulatory variation.
• Adaptive local position embeddings: The architecture uses position embeddings suited to variable-length promoter edits, supporting sequences up to 1,007 tokens.
• Benchmark suite: The authors release Ultra-Rare, eQTL, and transcription-factor binding site (TFBS) evaluation datasets for regulatory variant effect prediction.

#Technical Details

LOL-EVE is a 12-layer transformer with a hidden size of 768 and 12 attention heads, using a 39,378-token vocabulary and a maximum sequence length of 1,007 tokens with adaptive local position embeddings. It is trained on 13.6 million mammalian promoter sequences assembled into the PromoterZoo corpus. Across the Ultra-Rare, eQTL, and TFBS benchmarks, the model demonstrates strong performance in ranking regulatory variants relative to existing genomic sequence models. Code is released under the MIT license, and pretrained weights are distributed through the Hugging Face model hub, whose model card documents usage and benchmark analysis.

#Applications

LOL-EVE is aimed at researchers and clinical geneticists interpreting noncoding promoter variation. It can prioritize rare regulatory indels in developmental disorder genes, evaluate promoter variants of uncertain significance, and provide evolution-informed scores to complement functional genomics and eQTL evidence. Because scoring is zero-shot, the model can be applied to newly observed variants without retraining.

#Impact

By treating promoter indels as first-class objects and grounding predictions in mammalian evolutionary sequence, LOL-EVE extends the influential EVE line of evolution-based variant effect models from proteins into the noncoding regulatory genome. It addresses a class of variants that most genomic language models ignore, offering a route to more complete interpretation of regulatory variation in rare disease. As a preprint accompanied by open code, weights, and benchmarks, it provides a reusable foundation for downstream regulatory-variant work.