In-Field Phenotyping Based on Crop Leaf and Plant Instance Segmentation

Jan Weyler, Federico Magistri, Peter Seitz, Jens Behley, Cyrill Stachniss; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2022, pp. 2725-2734

A detailed analysis of a plant's phenotype in real field conditions is critical for plant scientists and breeders to understand plant function. In contrast to traditional phenotyping performed manually, vision-based systems have the potential for an objective and automated assessment with high spatial and temporal resolution. One of such systems' objectives is to detect and segment individual leaves of each plant since this information correlates to the growth stage and provides phenotypic traits, such as leaf count, coverage, and size. In this paper, we propose a vision-based approach that performs instance segmentation of individual crop leaves and associates each with its corresponding crop plant in real fields. This enables us to compute relevant basic phenotypic traits on a per-plant level. We employ a convolutional neural network and operate directly on drone imagery. The network generates two different representations of the input image that we utilize to cluster individual crop leaf and plant instances. We propose a novel method to compute clustering regions based on our network's predictions that achieves high accuracy. Furthermore, we compare to other state-of-the-art approaches and show that our system achieves superior performance. The source code of our approach is available.