Latent flow-matching model for wheat field imagery that jointly generates canopy images and organ masks over time, for segmentation and forecasting.
No providers recorded yet. Browse all providers
A wheat breeder who wants to track how leaves, stems and heads develop across a season needs two things that are each expensive on their own: pixel-level organ labels, which take hours of hand annotation per image, and repeated, registered photographs of the same plots through the season. Earlier generative models for plants covered only half of the problem: some forecast future images without structural labels, others synthesize labeled image-mask pairs for a single moment.
OneGrow, from Mike Boss at ETH Zurich's Institute of Agricultural Sciences, Michele Volpi of the Swiss Data Science Center and Lukas Roth of Martin Luther University Halle-Wittenberg, treats the problem as one joint distribution. It is a latent flow-matching model over wheat canopy images and their organ-segmentation masks across several time points. The key device is the reveal, a per-patch flag that marks which content is supplied as clean context and which the model must generate. Revealing images and generating masks is segmentation. Revealing masks and generating images is mask-conditioned synthesis. Revealing early frames and generating later ones is growth forecasting. Only the reveal changes between tasks, so a single trained checkpoint serves all of them without retraining.
The approach adapts unified "any view as context or target" generators from general computer vision, notably OneDiffusion and UniGS, to sparse seasonal field observations rather than densely sampled video. The preprint was posted to bioRxiv in September 2026.
OneGrow is a flow-matching diffusion transformer with width 512, 16 blocks, 8 attention heads and latent patch size 2. Every image and mask is encoded by the frozen Stable Diffusion 3.5 VAE (16 latent channels, eightfold downsampling), so each 512×512 view becomes 1,024 patch tokens. Tokens carry rotary position embeddings over modality, frame time and space. Revealed patches sit at flow time 1, all generated patches share one sampled flow time, and the loss is velocity regression over generated patches only.
Training draws with equal probability from two sources. The Global Wheat Full Semantic Organ Segmentation (GWFSS) challenge release contributes about 64,000 single-frame images from nine institutions. The FIP 1.0 dataset from ETH's Field Phenotyping Platform contributes four-frame windows from roughly 4,000 winter-wheat plots imaged over six years. Masks for both sources (background, head, stem, leaf) are pseudo-labels produced by the GWFSS challenge-winning ViT-Adapter/Mask2Former segmenter. On the 1,096 held-out human-annotated GWFSS images, OneGrow's image-to-mask mode reaches 0.697 mean IoU against 0.740 for that segmenter, with the largest gap on thin stems. Mask adherence, scored by re-segmenting images generated from masks, reaches 0.749 mean IoU. For mask-to-image synthesis, OneGrow gives lower paired LPIPS than a nearest-neighbour retrieval baseline on both datasets. The retrieval baseline scores better on set-level FD-DINOv2 and CMMD on GWFSS.
The main intended use is data augmentation for plant phenotyping. Given an organ mask, whether drawn from a real image, outpainted from a seed tile, or rendered from a functional-structural plant model, OneGrow produces a realistic canopy image whose labels are known by construction. That can widen segmentation training sets to growth stages and imaging conditions that are under-represented in real data. The temporal roll-out lets breeders and crop modellers turn a sequence of masks into a plausible image time series of a developing plot.
OneGrow unites two lines of plant generative modelling that had developed separately, organ-labeled image synthesis and temporal growth prediction, and it shows that the reveal formulation from general-domain unified generators carries over to sparse agricultural time series. Its scope is deliberately narrow. It is trained only on wheat, transfer to other crops is untested, and its forecasting and roll-out behaviour is demonstrated qualitatively rather than benchmarked. Because training masks come from a single pretrained segmenter, OneGrow inherits that teacher's biases, especially on stems, and it is slower at inference than a dedicated segmentation network. The work is a bioRxiv preprint that has not yet been peer reviewed, and it was released without code or trained weights.
Much of this page is generated or calculated automatically. Flag anything that looks off and we will re-run it.