DNABERT-Enhancer

Enhancers are cis-regulatory elements that boost transcription from a distance, and they are hard to find. They vary in length from roughly 100 to 1,500 bp, have fuzzy boundaries, act independently of orientation, and are active only in particular cell types, so no single signature cleanly separates them from the rest of the non-coding genome. Experimental mapping is limited by how many cell types can realistically be assayed, and the machine-learning predictors that fill the gap, including the iEnhancer family, mostly learn from nucleotide composition on a benchmark of fewer than 1,500 positive sequences.

DNABERT-Enhancer, from Ramana Davuluri's group at Stony Brook University with collaborators at the La Jolla Institute for Immunology and UC San Diego, takes the transfer-learning route instead. It fine-tunes DNABERT — the same lab's BERT-style genomic language model, which tokenizes DNA into overlapping k-mers — on a large enhancer collection curated from the ENCODE Registry of candidate cis-Regulatory Elements. It is a sibling to DNABERT-Epi: both are downstream checkpoints on the original DNABERT rather than on the multi-species successor DNABERT-2, but where DNABERT-Epi scores CRISPR off-targets, this one produces a genome-wide regulatory annotation and a catalog of variants predicted to break it.

The preprint was posted in March 2025, and both fine-tuned checkpoints are publicly archived, making the model an annotation resource rather than only a reported benchmark result.

#Key Features

• Two window lengths: Separate models were fine-tuned for 201 bp and 350 bp inputs, matching the two peaks in the length distribution of ENCODE cCRE enhancers.
• Genome-wide annotation: The best model was run over the entire GRCh38 assembly, and consecutive positive windows were merged into contiguous calls of arbitrary length.
• Allele-specific variant scoring: Predictions are recomputed on sequences carrying each alternate allele, and the score change and log odds ratio quantify how disruptive a SNP is.
• Transcription factor context: Running DNABERT models fine-tuned on ENCODE ChIP-seq peaks over the same variants identifies which binding site inside an enhancer a variant disrupts, with attention concentrating at and around the variant.

#Technical Details

Positives were drawn from the 961,227 enhancers in the ENCODE cCRE V3 registry for GRCh38: 21,926 of length 201 bp and 46,159 of length 350 bp. Negatives were sampled from the reference genome after excluding both cCRE enhancers and regions cataloged in ten public databases, then balanced 1:1 against positives and split 80/20 into training and held-out test sets. Sequences were tokenized as 6-mers, and fine-tuning ran on 8 NVIDIA A40 GPUs; the 350 bp model converged best at a learning rate of 3e-5, a 0.2 warm-up rate, and 0.0001 weight decay.

DNABERT-Enhancer-350 reached 88.05% accuracy, 90.27% precision, 85.29% recall, an F1 of 87.71%, and a Matthews correlation coefficient of 76.22% on held-out data, against 82.04% accuracy and 64.27% MCC for the 201 bp model — longer windows carry more regulatory context than the 200 bp convention inherited from nucleosome-length reasoning. Both models beat six classical classifiers trained on the same DNABERT embeddings, among them random forests, SVC, and AdaBoost, and outperformed iEnhancer-ELM, iEnhancer-DCLA, and iEnhancer-ECNN on accuracy, F1, and MCC. Applied to 19,582,268 subsequences tiled across the genome with a 150 bp stride, the model returned 1,822,321 merged enhancer regions covering about 22% of the genome; 79.06% of the ENCODE SCREEN catalog was recovered, 98.07% of the 1,036 experimentally validated VISTA enhancers were captured, and about 99% of the 1,308,887 calls outside SCREEN intersected at least one other public enhancer database.

#Applications

The direct output is a genome-wide enhancer annotation track extending beyond experimentally derived catalogs, useful for interpreting non-coding regions where no assay has been run. The variant analysis is the sharper application: 62,592 dbSNP variants were predicted to disrupt 5,069 SCREEN enhancers, and 458 high-accuracy DNABERT transcription factor models narrowed 15,693 of those to specific binding-site disruptions affecting 349 factors. Querying ClinVar and the GWAS catalog surfaced 84 variants with an associated trait, including a prostate cancer risk variant predicted to break a MAX binding site at PCAT19 and an atrial fibrillation-associated variant in an ATF4 site at HS1BP3. Cross-referencing GTEx cis-eQTLs yielded 143 high-confidence variants, 56 of them predicted to disrupt binding sites.

#Impact

The contribution is less architectural than practical: it shows that a genomic language model fine-tuned on a large, uniformly curated enhancer set generalizes far better than predictors trained on the small ChromHMM-derived benchmark the field has reused for years, and it ships the resulting annotations and candidate variants as a resource. Enhancers shorter than 350 bp are systematically missed, and the SCREEN regions the model failed to recover carry the nucleotide composition of non-enhancer sequence: lower C and G content and weaker CpG signal. Predictions come from sequence alone with no cell-type conditioning, so a call indicates enhancer potential rather than activity in a given tissue, and the prioritized variants are computational candidates awaiting experimental testing. Code is on GitHub under Apache-2.0 and both fine-tuned checkpoints are archived on Zenodo.