Deep Learning Based 3d Reconstruction for Phenotyping of Wheat Seeds: a Dataset, Challenge, and Baseline Method

Vsevolod Cherepashkin, Erenus Yildiz, Andreas Fischbach, Leif Kobbelt, Hanno Scharr; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 561-571

We present a new data set for 3d wheat seed reconstruction, propose a challenge, and provide baseline methods. Individual plant seed properties influence early development of plants and are thus of interest in plant phenotyping experiments. Seed shape can be measured reliably from images using volume carving, as done in robotic setups such as phenoSeeder. However, about 36 images are needed to obtain a suitably accurate 3d model, where image acquisition takes approximately 20 s. For large-scale experiments with thousands of seeds higher throughput is required limiting image acquisition time. We present a deep-learning model that reconstructs an approximate 3d point cloud from fewer images, even only a single view. It has a significantly lower error than linear regression, which has been actively used so far in similar tasks. Using three images reduces imaging time by a factor of 10, where relative errors of volume length, width, and height are all around 2 %. Inference time from the neural network is negligibly short compared with imaging time which enables this method for real-time measurements and sorting.