Visual 3D Reconstruction and Dynamic Simulation of Fruit Trees for Robotic Manipulation

Francisco Yandun, Abhisesh Silwal, George Kantor; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2020, pp. 54-55

Modern agriculture is facing a series of challenges to adopt new technologies to improve sustainability, profitability and resilience. One of them is the use of robotic applications to assist or even replace manual workers for the complex task of interaction with the vegetation. For example, harvesting and pruning are tasks that need certain dexterity to not only make the cuts, but also to move branches or foliage in the canopy to reach hidden objects or locations. For such capability, first the robot should be able to perceive the vegetation and estimate the dynamics for the interaction. This work mainly focuses on the perception problem, aiming to digitize commercial tree fruit canopies and estimating how it moves when force is applied to the branches. We studied the suitability of two known algorithms, viz. the space colonization and the Laplace based contraction algorithms, to build a geometric model of the tree using point cloud data from stereo cameras. Such model is then used to estimate the dynamics of the tree, by considering the branches as links articulated by spring-damper joints. The geometric model was evaluated for topological and morphological correctness by comparing it with the ground truth, obtaining better results with the Laplace based contraction algorithm. Furthermore, results of the dynamics estimation showed that by adjusting the parameters for the spring-damper model, the motion prediction is promising, with a maximum mean squared error of 0.073m in the tracking of the movement of the branches.