Vision-Based Monitoring of the Short-Term Dynamic Behaviour of Plants for Automated Phenotyping

Nikolaus Wagner, Grzegorz Cielniak; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 624-633

Modern computer vision technology plays an increasingly important role in agriculture. Automated monitoring of plants for example is an essential task in several applications, such as high-throughput phenotyping or plant health monitoring. Under external influences like wind, plants typically exhibit dynamic behaviours which reveal important characteristics of their structure and condition. These behaviours, however, are typically not considered by state-of-the-art automated phenotyping methods which mostly observe static plant properties. In this paper, we propose an automated system for monitoring oscillatory plant movement from video sequences. We employ harmonic inversion for the purpose of efficiently and accurately estimating the eigenfrequency and damping parameters of individual plant parts. The achieved accuracy is compared against values obtained by performing the Discrete Fourier Transform (DFT), which we use as a baseline. We demonstrate the applicability of this approach on different plants and plant parts, like wheat ears, hanging vines, as well as stems and stalks, which exhibit a range of oscillatory motions. By utilising harmonic inversion, we are able to consistently obtain more accurate values for the eigenfrequencies compared to those obtained by DFT. We are furthermore able to directly estimate values for the damping coefficient, achieving a similar accuracy as via DFT-based methods, but without the additional computational effort required for the latter. With the approach presented in this paper, it is possible to obtain estimates of mechanical plant characteristics in an automated manner, enabling novel automated acquisition of novel traits for phenotyping.