Pseudo-Label Generation for Agricultural Robotics Applications

Thomas A. Ciarfuglia, Ionut Marian Motoi, Leonardo Saraceni, Daniele Nardi; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022, pp. 1686-1694

In the context of table grape cultivation there is rising interest in robotic solutions for harvesting, pruning, precision spraying and other agronomic tasks. Perception algorithms at the core of these systems require large amounts of labelled data, which in this context is often not available. In this work, we propose a semi-supervised solution to reduce the data needed to get state-of-the-art detection and segmentation of fruits in orchards. We present the case of table grape vineyards in southern Lazio (Italy) since grapes are a difficult fruit to segment due to occlusion, color and general illumination conditions. We consider the concrete scenario where the source labelled data is wine grape, while the target data is table grape, with considerable covariate shift. Starting from a simple video input, our method generates first bounding box labels, leveraging the structure from motion information, then segmentation masks, using the same weakly generated bounding box labels and a refining step based on Grab Cut. This system is able to produce labels that considerably reduce the covariate shift from source to target data and that requires very limited data acquisition effort. Comparisons with SotA supervised solutions show how our methods are able to train new models that achieve high performances with few labelled images, with very simple labelling.