DNT

Two pathogenic variants in the same recessive gene mean different things depending on where they sit. In trans — one on each homologue — both copies are disrupted and the person is affected; in cis one intact copy remains. The variant pair is identical either way and only the arrangement differs, so a model reading a single haploid sequence receives the same input twice and cannot, in principle, tell them apart.

DNT closes that gap by writing the diploid genome as one sequence. The maternal and paternal homologues are aligned column by column and emitted as a single token stream, one token per aligned position: homozygous columns keep the ordinary A/C/G/T alphabet, while heterozygous columns become an atomic allele-pair symbol, so both alleles reach the encoder in one forward pass. Short indels are rendered inside explicit span boundaries with allele-specific gap padding, preserving inserted and deleted bases as sequence rather than collapsing them into a generic event token. Zygosity, allele dosage and — where phasing is supplied — the assignment of alleles to homologues therefore become properties of the input rather than something a downstream head must reconstruct.

The work comes from Sheba Medical Center with NVIDIA and the Icahn School of Medicine at Mount Sinai, posted to bioRxiv in September 2026 by largely the same collaboration behind JEPA-DNA. It does not pretrain from scratch: it continues training NTv3 backbones on diploid-encoded human sequence.

#Key Features

• Reference-aligned diploid encoding: One token per aligned reference column carries the allelic state of both homologues, keeping the genotype inside the sequence.
• Three tokenizers over one vocabulary: An unphased tokenizer maps reciprocal allele pairs to the same symbol; an ordered-phase tokenizer distinguishes the two orders; the primary marker-phase tokenizer pairs an unordered symbol with an orientation marker. All three consume supplied phasing — none estimates phase.
• Contrastive Phase Loss: An auxiliary objective trained alongside masked language modelling that penalises representations invariant to a homologue swap, forcing orientation to survive contextual encoding.
• Vocabulary grafted onto a pretrained backbone: 44 diploid rows are added to NTv3's 11-token base vocabulary, each new embedding seeded from the nucleotides it composes and warmed up for 5,000 steps with the backbone frozen.
• A benchmark that isolates phase: In the compound-heterozygous suite, cis and trans examples draw the same variants from the same genes in the same individuals, so variant identity and gene content cannot explain a score.

#Technical Details

Two scales were continually pretrained, from the 8M and 100M Nucleotide Transformer v3 checkpoints, on reference-aligned diploid sequence from the high-coverage phased 1000 Genomes Project release. The interval set — GENCODE v47 basic coding exons merged with UCSC 100-way phastCons elements — comprises 140,099 regions spanning roughly 943 Mb, about 30% of GRCh38; training used 4,096-token windows and split at the individual and pedigree level. The compound-heterozygous benchmark injects curated pathogenic variants into individual 1000 Genomes haplotype backgrounds across a 117-gene panel, yielding 9,460 family-aware examples split 6,669 / 980 / 1,811. Phase-blind conditions sit at chance there — 0.491 to 0.533 AUROC — while the marker-phase full-epoch models trained with Contrastive Phase Loss reach 0.613 and 0.624 frozen and 0.649 and 0.639 fine-tuned at 8M and 100M. Separately, reading one trained model through the diploid encoding rather than a haploid one raises supervised ClinVar variant-effect AUROC in 10 of 11 arms by a mean of 0.106, peaking at 0.853; zero-shot ClinVar indel scores peak at 0.728. GFMBench-API general-capability controls show the expanded vocabulary mostly preserves haploid performance — the diploid checkpoint beats its NTv3 baseline in 36 of 54 benchmark-metric comparisons at 8M and 43 of 54 at 100M, with SongLab ClinVar the clearest regression. Scale is not monotonic: the 8M models beat their 100M counterparts in several comparisons.

#Applications

The clinically interesting part is the class of questions the encoding can express: carrier status, biallelic disease, hemizygosity and compound heterozygosity all turn on genotype configuration and cannot be reduced to an isolated alternative allele. A genotype-aware representation is a component for inheritance-constrained variant prioritisation and for folding phase into rare-disease workflows. A zero-shot case study on CFTR separates ClinVar-annotated pathogenic from benign SNVs at 0.780 AUROC, with distinct local score profiles for heterozygous and homozygous F508del.

#Impact

DNT's contribution is the representation, and the authors are explicit that this is a method for making diploid genotype information accessible to genomic language models rather than a universal improvement in variant prediction. Several limits bound it. The compound-heterozygous examples are synthetic and the phasing is statistical, so validation in naturally observed, accurately phased clinical cohorts remains to be done. The encoding covers SNVs and short indels only, not structural variants, copy-number changes or repeat expansions, and training used 4,096-token windows even though the backbone supports far longer contexts. Every experiment runs on NTv3, so generalisation to other architectures is untested. The checkpoints are published on HuggingFace under a research-only licence with gated access, and the repository named in the paper is live but has not yet posted its code.