3D Point Cloud Instance Segmentation of Lettuce Based on PartNet

Luhan Wang, Lihua Zheng, Minjuan Wang; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022, pp. 1647-1655

Organ level instance segmentation (e.g., individual leaves) based on computer vision techniques is a key step in the measurement of plant phenotypes. Since plant organs, especially leaves, are self-occluded and emerged-occluded, single-view images affect the acquisition of some effective information. However, 3D global images contain much more plant morphological information than single-view images, and it is of great significance for plant phenotype research. In this paper, lettuce was taken as the research object, its 3D point cloud images were obtained and instance segmentation was carried out based on the deep learning method. The result showed that the 3D point cloud of each leaf was segmented and identified accurately. Specifically, we constructed a lettuce point cloud dataset consisting of 620 real and virtual point clouds and fused them together to train a 3D instance segmentation network--PartNet, which directly takes 3D point clouds as input and its output is the instance segmentation results of leaves. The experimental results showed that, when tested with 40 point clouds in the validation set, the instance segmentation accuracy AP (%) with IoU < 0.25 reaches 97.2%, and the instance segmentation accuracy AP with IoU < 0.5 reaches 92.4%, indicating that the constructed PartNet network has the potential to accurately segment the 3D point cloud leaf instances for lettuce.