Tracking and Counting Apples in Orchards Under Intermittent Occlusions and Low Frame Rates

Gonçalo P. Matos, Carlos Santiago, João P. Costeira, Ricardo L. Saldanha, Ernesto M. Morgado; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 5413-5421

Estimating what will be the fruit yield in an orchard helps farmers to better plan the resources needed for harvesting storing and commercialising the crop and also to take some agricultural decisions (like pruning) that may increase the quality of the yield and increase profits. Therefore over the last years several methods based on computer vision were proposed to automate this task by directly counting the fruits on trees using a video camera. However existing works and methods usually assume ideal conditions and may fail under more challenging scenarios with unconstrained camera motion and intermittent occlusions of fruits. Here we show that combining SfM with a bipartite graph matching has the potential to address those challenges. We found that our approach applied to real-world datasets with unconstrained camera motion and low frame rates outperforms existing methods by a large margin. Our results demonstrate that the proposed method is robust to multiple intermittent occlusions under challenging conditions and thus suitable to be used in diverse real-world scenarios in orchards either with a camera operated by hand or mounted on an agricultural vehicle. Although not shown here we believe that the proposed method can also be applied to other object tracking problems besides counting fruits under similar settings --- i.e. static objects and a freely moving camera.