LLMED

Edit distance — the minimum number of substitutions, insertions, and deletions that turn one string into another — is the elementary metric for comparing genomic sequences, but computing it exactly takes quadratic time, ruling it out for all-against-all comparison at scale. The standard workaround is edit distance embedding: map each sequence to a fixed-length vector so that a cheap geometric distance in that space approximates the edit distance between the original strings. Rule-based schemes such as Tensor Sketch and learned encoders such as CNNED and NeuroSEED compete on the same benchmarks, and approximation accuracy has remained the binding constraint.

LLMED, from Mingfu Shao's group at Pennsylvania State University with a collaborator at Lanzhou University, asks whether a pretrained DNA language model can close that gap. It takes DNABERT-2 as its backbone, average-pools the per-token hidden states into a single sequence embedding, and fine-tunes the encoder with contrastive objectives that tie cosine similarity in the embedding space directly to edit similarity between the input sequences. The result is a fixed encoder: sequences pass through it once, and downstream search or clustering runs on the resulting vectors without per-dataset retraining.

The work is notable less for a new architecture than for the target it points an existing one at. Genomic foundation models such as DNABERT-2, Nucleotide Transformer, and Evo are normally fine-tuned for biological prediction — binding sites, splice junctions, variant effects. LLMED instead trains one to reproduce a string metric, treating the language model as a component in a classical algorithms problem. The preprint was posted to bioRxiv in September 2025 and awaits peer review.

#Key Features

• Edit distance as the explicit training signal: The loss ties cosine similarity between two embeddings to a length-normalized edit similarity, calibrating the output geometry to the metric it stands in for.
• Three contrastive objectives: Separate checkpoints are trained with a mean-absolute-error loss, a triplet loss over anchor/positive/negative sequence triples, and a weighted combination of the two.
• Training-free distance readout: The MAE objective fixes the relationship between cosine and edit similarity, so edit distance follows from a closed-form expression over the cosine similarity and the two sequence lengths, with no calibration set.
• Backbone-agnostic framework: The pooling-plus-contrastive-fine-tuning recipe is defined over an arbitrary genomic language model; DNABERT-2 is the instantiation reported, not a requirement of the method.
• Open code and weights: The code and the LLMED-MAE checkpoint are released under BSD-3-Clause, with evaluation data archived on Zenodo under CC-BY 4.0. The triplet and combined checkpoints are published without a stated license.

#Technical Details

LLMED is built on the 117M-parameter DNABERT-2 checkpoint, inheriting its byte-pair-encoding tokenizer and 768-dimensional hidden states; the sequence embedding is the mean over all token positions. The MAE variant is fine-tuned on one million simulated sequence pairs — a random string over {A, C, G, T} paired with a mutated copy at a mutation rate drawn uniformly from 3% to 30% — split 95/5, for 10 epochs at a learning rate of 1e-5 and batch size 16. The triplet variant draws positives at a 10% mutation rate and negatives at 40% with margin 0.3; the combined variant weights the MAE and triplet terms at 0.5 and 1.

On simulated pairs from 100 to 1,000 nucleotides, every LLMED variant exceeds 0.95 Spearman correlation with true edit distance at length 1,000, ahead of Tensor Sketch and CNNED, while the unmodified DNABERT-2 checkpoint correlates poorly — the gain comes from the fine-tuning, not the pretrained representation. The combined-loss variant attains the lowest mean squared error under both the fitted-linear and closed-form readouts. In top-K nearest-neighbor search, the three variants match Tensor Sketch on a synthetic 55,000-sequence set; on Gen50ks, 50,000 real fragments averaging 5,000 bp from human chromosome 20, LLMED-triplet leads on hit ratio and recall while Tensor Sketch degrades noticeably.

#Applications

The immediate use is any pipeline needing approximate edit distance at scale: similar-sequence search over large read or fragment collections, alignment-free similarity estimation, sequence clustering, and phylogeny reconstruction, for which the repository includes a workflow that builds trees from embedding distances and scores them by normalized Robinson-Foulds. Since no per-dataset retraining is required, the embeddings drop into an existing indexing or nearest-neighbor stack in place of a sketching step.

#Impact

LLMED is an early demonstration that genomic language models can be repurposed for algorithmic primitives rather than only biological prediction, and that a 117M-parameter encoder fine-tuned on synthetic mutations can outperform sketching methods purpose-built for the task, most visibly on real genomic data. The limitations are equally clear: the model is optimized for a single geometric property, transfer to other genomic tasks is not demonstrated, the training distribution is a synthetic mutation model rather than observed sequence variation, and running a transformer over every sequence costs far more per comparison than computing a sketch. As a preprint from a single group with a small release footprint, its influence on production tooling remains to be established.