Plant Root Occlusion Inpainting with Generative Adversarial Network

Hao Song, Karim Panjvani, Zhigang Liu, Huzaifa Amar, Leon Kochian, Shengjian Ye, Xuan Yang, J. Allan Feurtado, Krunal Chavda, Karina Angela Chimbo Huatatoca, Mark Eramian; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 531-539

Three-dimensional (3D) image analysis represents the state-of-the art for phenotyping in the fields of biology and plant science including studies of root system architecture. A widely used approach for capturing root architecture in 3D involves growth of roots in hydroponic media and capture of optical camera views via a stepper-motor-based rotation system. However, the introduction of structures to support 3D root growth system leads to significant occlusion of the roots during image acquisition, thereby causing the complexity and introducing inaccuracy of subsequent operations such as 3D modeling and root traits calculation. Instead of using a traditional manual sketching methods, this project proposes an automatic root gaps detection and inpainting method based on a Generative Adversarial Networks (GAN). The model was trained and evaluated using two distinct maize datasets, both of which were enriched with manually annotated segmentation and inpainting labels. The quantitative analysis of the inpainting results demonstrated variation in the performance of the GAN model. However, promising outcomes were observed with certain instances achieving Intersection of Union (IoU) and Dice Similarity Coefficient (DSC) values surpassing 0.9 with specific images or patches exhibiting lower accuracy and reproducibly. Despite this variability, the overall model performance maintained an average range of 0.8-0.9. Our GAN model presents a robust, effective and automatic solution for inpainting plant root gaps, leading to improved accuracy within the phenotyping pipeline. Moreover, the model demonstrates a great generality for inpainting other root system of species or cultivars beyond those encountered during training. The performance of the model exhibits superiority when confronted with less intricate root structures, but it produces less accurate results when confronted with complex root systems with large gaps or high root density.