Loading…
Genomic language model for tomato pangenomes that conditions DNA tokens on graph node attributes and local adjacency as well as masked modelling.
No providers recorded yet. Browse all providers
A tomato genome is usually handed to a sequence model as one long string, so a segment carried by all 66 accessions in a panel and a segment found in exactly one of them look like the same kind of thing: letters in a row. The difference matters — presence/absence and structural variation absent from the SL6.0 Heinz 1706 reference carry alleles for fruit quality, stress response and breeding traits. A graph pangenome stores that as node attributes and edges, and it is discarded the moment assemblies are flattened into separate sequence documents.
TomatoPGFM, from the Xinjiang Academy of Agricultural Sciences and Nanjing Agricultural University, keeps nucleotide sequence as the primary input and supplies graph structure alongside it as an explicit conditioning signal. Each token carries an eight-channel, token-aligned view of the graph segment it falls in, and hidden states also receive mean-aggregated messages from incoming within-window neighbours. Pretraining asks the model both to fill in masked tokens and to reconstruct those graph channels, so the graph is a prediction target rather than a side input the network can ignore.
OneGenome-Rice and OryzaG3 widen the pretraining corpus by treating many assemblies as additional sequence documents; TomatoPGFM instead injects graph coordinates into the encoder. Because the conditioning arrives through gated residual pathways, a zero-feature mode lets the same checkpoint run purely on sequence when no graph mapping is available.
is_random_only flag — through a gated residual
pathway. Five of the eight varied in the production pretraining shards.The pretraining panel covered 66 accessions spanning cultivated Solanum lycopersicum sensu lato and seven wild relatives, totalling 54.65 Gb of chromosome-scale assembly. Using SL6.0 Heinz 1706 as the backbone, minigraph produced 3,203,440 segments and 4,450,064 links over 2.598 Gb of graph sequence; 170,034 segments were traversed by all 66 accession paths while 506,198 sat on one. Sequence is tokenised as non-overlapping, reverse-complement-folded 6-mers and fed to a 15-block hybrid backbone of 10 bidirectional Mamba2 blocks and five local-attention mixture-of-experts blocks, totalling 479,195,678 unique parameters with roughly 148,625,438 active per token. A five-stage curriculum raised model length from 512 to 8,192 positions across 116,883 optimisation steps, completing in 46.09 hours on four 80-GB A100 GPUs.
Two evaluations answer different questions. On 25,000 training-panel windows, sequence-aligned graph input gave lower masked-modelling loss than graph-off at all five curriculum stages, with shuffled node features landing in between — a sensitivity test, not an isolation of any feature group. Separately, S. sitiens LA1974 and the S. lycopersicum cultivar MicroTom, used in neither graph construction nor pretraining, were classified gene- and CDS-versus-intergenic from frozen mean-pooled hidden states with graph input off. AUROC ran from 0.8489 to 0.9593, above DNABERT-2 by 0.0387 to 0.0531 in this paper's own four comparisons but below PlantDNAMamba in all four; LoRA adaptation preserved the same ordering.
The practical target is tomato genome annotation and breeding genomics, where dozens of accession assemblies now exist and a reference-anchored model under-represents what distinguishes them. Frozen embeddings separate genic and coding windows from intergenic sequence in accessions the model never saw, making the checkpoint usable as a feature extractor for annotation triage on newly assembled germplasm, and rank-8 LoRA adapts roughly 1.2 million parameters for task-specific fitting. The graph pathway serves groups that have mapped their material into a pangenome graph; the sequence-only path serves everyone else.
TomatoPGFM tests a specific proposition: that population-level graph context belongs inside a genomic foundation model's encoder rather than only in its corpus. The authors are careful about what they have shown. Graph conditioning was demonstrated on training-panel windows, while the cross-accession result used the sequence-only path and no matched sequence-only pretraining control was trained, so the two findings establish prerequisites rather than a claim that graph conditioning generalises to new accessions. The classification tasks also do not exercise the variation the graph encodes; accession-dependent gene presence and structural-variant-associated expression would. The work is a preprint awaiting peer review, and the release covers inference only — the optimizer-bearing training checkpoint is deliberately withheld.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.