Reinforcement Learning with Space Carving for Plant Scanning

Antonio Pico Villalpando, Matthias Kubisch, David Colliaux, Peter Hanappe, Verena V. Hafner; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 694-701

Optimal plant reconstruction is an essential element in automating our future agriculture. Computerized inspection of proper growth, nutrition, or pest infestation has become mandatory in fully autonomous in-door or micro-farm settings, shifting from fixed to moving camera systems. In industrial environments, plant scanning must work efficiently with a limited number of significant images to become economically viable. We present an adaptive learning algorithm for agricultural plant inspection robots, in particular, a specific type of reinforcement learning that we developed for our micro-farming platform created within the EU project ROMI. We suggest a new approach to 3D plant reconstruction by integrating the space carving technique with categorical Deep Q-Networks. Space carving leverages images captured from various positions to create a binary voxel grid, representing the occupied and unoccupied spaces of the scanned object. The proposed method incorporates partial 3D reconstructions of plants obtained through space carving, which get compared to a ground truth model to calculate the reward and guide scanning policies. We explain the algorithmic details and the 3D reconstruction technique in design, implementation, and evaluation. Experimental results confirm our approach's effectiveness in improving the 3D plant reconstruction process, highlighting its potential for further applications in agriculture and related fields.