TrinityDNA

TrinityDNA is a 1-billion-parameter DNA foundation model built around three architectural ideas drawn from the physical structure of the double helix: convolutional tokenization sized to DNA's major and minor grooves, gating over the reverse-complement strand, and attention heads with different window widths so local motifs and kilobase-scale dependencies are modeled together. It was developed by BioMap Research with the AI Lab at Westlake University, released as a preprint in July 2025 and accepted to AAAI 2026.

The model targets a specific failure mode in genomic sequence modeling. State space models such as HyenaDNA and Caduceus scale to very long contexts but carry a locality bias: the influence of an input base on a distant output decays sharply as the sequence lengthens. Full-attention encoders such as DNABERT-2 avoid that bias but are quadratic in sequence length — DNABERT-2 runs out of memory at 16kb in the same analysis — and their attention flattens toward uniformity as context grows, an oversmoothing effect that is especially damaging for DNA, where informative bases are sparse and far apart.

TrinityDNA's answer is a transformer encoder with structurally informed inductive biases plus a two-stage "evolutionary" curriculum that begins on prokaryotic genomes and then extends to eukaryotes while growing the context window. The paper describes two 1B checkpoints: TrinityMicroDNA, trained on prokaryotes only, and TrinityDNA, which continues pretraining on multi-species data.

#Key Features

• Groove Fusion tokenization: Parallel convolutions with kernel sizes 3, 5, and 7 followed by GELU activations produce the input embedding, matching the roughly 3–5 and 5–7 nucleotide spans of DNA's minor and major grooves. Ablation shows a 0.065 reduction in pretraining perplexity.
• Gated Reverse Complement: A shared encoder processes both the forward strand and its flipped reverse complement, combining the two representations through a learned gate. This is the largest single architectural contribution in the ablation, lowering perplexity by 0.132.
• Sliding multi-window attention: Each attention head is given its own sliding-window width, so some heads specialize in short-range motifs while others span the full context. This cuts pretraining cost from 64.5 to 44.5 TFLOPs, a 31% reduction, at essentially unchanged perplexity.
• Evolutionary training strategy: Stage one learns nucleotide grammar from prokaryotic genomes at 8kb; stage two adapts to eukaryotic intron-exon structure while the context window expands to 30kb and then 102,400 base pairs. Initializing from the prokaryotic checkpoint beats training on the combined corpus from scratch.
• CDS annotation benchmark: A long-sequence coding-sequence annotation task built from RefSeq prokaryotic reference genomes, with token-level CDS and strand-direction labels over 20kb windows.

#Technical Details

TrinityDNA is a transformer encoder trained with masked language modeling at single-nucleotide resolution: a five-token vocabulary (A, T, C, G, N), 15% masking, GEGLU activations, DeepNorm, RoPE positional encoding with dynamic NTK scaling, and FlashAttention-2. The 1B configuration uses 24 layers, hidden size 2048, FFN size 5461, and 32 heads; smaller configurations from 6M to 470M parameters were trained for the scaling analysis on 31 nodes of 8 A100 GPUs. Stage-one data is the OpenGenome corpus — bacterial and archaeal genomes from GTDB v214.1, prokaryotic viruses from IMG/VR v4, and plasmids from IMG/PR — and stage two uses the multi-species corpus from the Nucleotide Transformer work.

On the GUE benchmark with LoRA fine-tuning, TrinityDNA-1B reaches an average MCC of 0.708 across 15 tasks, ahead of the 2.5B Nucleotide Transformer (0.636), DNABERT-2 (0.621), HyenaDNA (0.610), and DNABERT (0.552), with the largest gains on mouse transcription factor binding (0.786) and splice site reconstruction (0.927). Across 19 zero-shot tasks, TrinityMicroDNA-1B leads the prokaryotic set (average 0.475), while TrinityDNA-1B leads the eukaryotic set (0.699), above the 40B Evo 2 (0.667), and reaches 0.933 AUC on ClinVar pathogenicity. On the new CDS benchmark, TrinityMicroDNA-1B achieves an exact-match F1 of 0.754 against 0.725 for Prodigal and 0.702 for GENSCAN, though Prodigal retains higher recall.

#Applications

The two checkpoints target different regimes. TrinityMicroDNA suits microbial genomics — gene-structure annotation of newly sequenced bacterial and archaeal genomes, where its token-level CDS predictions compete with the classical Prodigal and Glimmer pipelines without organism-specific parameterization. TrinityDNA targets eukaryotic and human genomics: promoter and enhancer detection, transcription factor binding site prediction, splice site identification, histone mark prediction, and zero-shot scoring of clinical variants. The 100kb context suits regulatory questions spanning multiple co-expressed genes rather than single elements.

#Impact

TrinityDNA is a case for architectural priors over raw scale in genomic modeling: a 1B model matching or exceeding a 40B Evo 2 on eukaryotic zero-shot fitness prediction and a 2.5B Nucleotide Transformer on supervised benchmarks. Its CDS annotation benchmark also fills a gap, since most DNA model evaluations use short classification tasks that never test long-range inference. The main practical limitation is availability: no code or weights have been released, and the preprint carries a CC BY-NC-SA 4.0 license, so the results cannot yet be independently reproduced or the checkpoints applied to new genomes.