Nucleotide Transformer v3 (NTv3)

Nucleotide Transformer v3 (NTv3) is a multi-species genomics foundation model from InstaDeep that unifies three capabilities previously split across separate systems: representation learning, functional-track and genome-annotation prediction, and controllable sequence generation. Introduced in a December 2025 bioRxiv preprint, it is the third generation of the Nucleotide Transformer line and a substantial redesign rather than an incremental update. Where the earlier encoder-only NT models tokenized DNA into overlapping six-mers and read 6–12 kb of context, NTv3 operates at single-nucleotide resolution over windows up to 1 megabase.

The core problem NTv3 targets is long-range genomic regulation. Enhancers, insulators, and other regulatory elements can act over hundreds of kilobases, a range invisible to short-context models. NTv3 reaches this scale with a U-Net-style architecture: a convolutional tower downsamples the 1 Mb single-base input, a transformer stack models dependencies in the compressed representation, and a deconvolutional path restores base-level resolution for both prediction and generation. This lets a single model move from raw sequence to dense functional signal without the fixed short windows of prior genomic language models.

NTv3 was developed by InstaDeep (London) with academic collaborators at the Research Institute of Molecular Pathology (IMP) and the Medical University of Vienna, Cornell Tech, and Cold Spring Harbor Laboratory. The model family, benchmark, and JAX code are released publicly under a non-commercial license.

#Key Features

• 1 Mb single-base context: NTv3 reads up to one megabase of DNA at nucleotide resolution, replacing the six-mer tokenization and 6–12 kb windows of earlier Nucleotide Transformer generations with character-level tokens over A, T, C, G, and N.
• U-Net plus transformer architecture: A convolutional downsampling tower compresses the long input, transformer layers model long-range dependencies in the reduced space, and deconvolution restores base resolution, making 1 Mb contexts computationally tractable.
• Joint sequence-to-function post-training: The model is post-trained on roughly 16,000 functional genomic tracks and genome-annotation labels across 24 animal and plant species, so one backbone predicts dense regulatory signal rather than only learning representations.
• Controllable enhancer generation: Fine-tuned into a generative model via masked diffusion language modeling, NTv3 designs enhancers with specified activity; 1,000 designed sequences were experimentally validated by STARR-seq, achieving more than twofold improved promoter specificity over baselines.
• Open model family and intermediate checkpoints: Pretrained (8M, 100M, 650M) and post-trained (100M, 650M) checkpoints are released on HuggingFace, along with research checkpoints at 8 kb and 131 kb context lengths.
• Standardized long-range benchmark: The accompanying NTv3 Benchmark provides 106 standardized sequence-to-function tasks for evaluating long-range genomic models across species.

#Technical Details

NTv3 is pretrained via masked language modeling on approximately 9 trillion base pairs drawn from the OpenGenome2 corpus, spanning more than 128,000 species, and is implemented in JAX for GPU and TPU execution. The largest released model has about 650M parameters (0.7B), with a 1,536 hidden dimension, 12 transformer layers, 24 attention heads, and a seven-stage convolutional downsampling tower; input length must be a multiple of 128. After pretraining, a joint objective post-trains the model on roughly 16,000 functional tracks (BigWig signal and BED elements) across 24 animal and plant species. On the NTv3 Benchmark of 106 long-range tasks, the authors report state-of-the-art accuracy for functional-track prediction and cross-species genome annotation.

#Applications

NTv3 is aimed at researchers in regulatory and comparative genomics who need to relate long stretches of sequence to function: predicting chromatin and expression signal, annotating regulatory elements in newly assembled genomes, scoring the effects of non-coding and structural variation, and designing synthetic regulatory sequences. The controllable generation capability is particularly relevant to synthetic biology and gene-therapy vector design, where enhancers with tuned, cell-type-specific activity are valuable. Released checkpoints across three sizes let groups trade accuracy against compute.

#Impact

NTv3 marks the Nucleotide Transformer line's shift from a representation-only backbone toward a unified sequence-to-function platform, joining the class of long-range genomic predictors that model regulatory signal at scale. Its single-base 1 Mb context directly addresses the short-window and six-mer limitations of NT v1 and v2, and its experimentally validated enhancer design demonstrates that a genomics foundation model can both read and write regulatory sequence. As a recent preprint, its results await peer review and independent replication, and the non-commercial license on the weights, code, and paper restricts commercial reuse and derivative models.