A 1B-parameter DNA language model pretrained on 48 crop and plant genomes to predict regulatory activity, gene expression and variant effects.
No providers recorded yet. Browse all providers
Most of what deep learning knows about gene regulation was learned from human and mouse. Plant genomics has the opposite problem: more than 200 plant genomes have been sequenced, but only a handful of model species such as Arabidopsis thaliana, rice and maize carry the tissue-level expression atlases, chromatin maps and reporter assays that supervised models need. A crop like cassava has a reference genome and little else. AgroNT (Agronomic Nucleotide Transformer) addresses that imbalance by learning the grammar of plant DNA from unlabeled reference genomes of 48 species, most of them edible crops, so that one pretrained model can be adapted to a regulatory task in any of them with far less labeled data.
AgroNT was developed by InstaDeep with Google DeepMind, posted to bioRxiv in October 2023 and published in Communications Biology in July 2024. It is a plant-specific sibling of InstaDeep's Nucleotide Transformer, sharing its encoder design and 6-mer tokenization but trained on a plant corpus. The paper also assembles the Plants Genomic Benchmark (PGB), a collection of plant regulatory tasks released alongside the model.
AgroNT is a BERT-style encoder-only transformer with one billion parameters: 40 attention blocks, a 1,500-dimensional embedding, learned positional embeddings and a RoBERTa language-model head. DNA is split into non-overlapping 6-mers over a vocabulary of about 4,100 tokens, so the 1,024-token context covers roughly 6 kb. The pretraining corpus is about 10.5 million sequences drawn from Ensembl Plants reference genomes. The model was trained with 15% token masking for 315,000 steps at an effective batch of 1.5 million tokens, 472.5 billion tokens in total, over about four days on a Google TPU v4-1024 slice.
After fine-tuning, AgroNT reaches AUROC 0.89–0.96 for polyadenylation sites across five species and 0.97 and 0.95 for splice acceptors and donors in Arabidopsis, and AUROC 0.94–0.98 for chromatin accessibility across six species and their tissues. For STARR-seq promoter strength it reaches R² of 0.70 in maize protoplasts and 0.73 in tobacco leaves, on average 3 R² points above a CNN built for that dataset. For terminators it reaches 0.67 and 0.77. Tissue-specific expression prediction from promoter-proximal sequence reaches R² of 0.42–0.62 across five species. In zero-shot variant scoring on rice, AgroNT's scores give odds ratios for very rare, likely deleterious variants up to eight times higher than those of GPN, a model pretrained on Arabidopsis and seven other Brassicales species. On Arabidopsis, where GPN was trained on close relatives, GPN and phyloP score higher.
AgroNT is aimed at plant geneticists and crop breeders who need to rank non-coding variants, find regulatory elements or design synthetic promoters and terminators in species that lack large functional datasets. Its fine-tuned heads predict promoter and terminator activity for construct design, open chromatin across tissues, and expression levels from promoter sequence. The zero-shot scores can prioritize candidate mutations before CRISPR editing experiments. The released cassava predictions show the workflow at genome scale: fine-tuned enhancer and expression models flagged individual substitutions in DOF and Trihelix transcription-factor motifs as the most disruptive.
AgroNT carried the Nucleotide Transformer recipe from human and multi-species genomes into crops, and its tasks became reference points for the plant sequence models that followed. PlantBiMoE is evaluated on a modified version of the Plants Genomic Benchmark and reports AgroNT as its main baseline. InstaDeep's ChatNT folds AgroNT's plant tasks into its instruction-tuning data. The 1B checkpoint is on HuggingFace with a PyTorch model card, and JAX inference code is in the Nucleotide Transformer repository. Both the weights and the benchmark are released under CC BY-NC-SA 4.0, which restricts them to non-commercial use, and the pretraining code is proprietary and was not released. The authors point to the 6 kb context as a limit on capturing distal elements such as enhancers and silencers, which longer-context models such as Enformer and HyenaDNA are built to capture.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.