ST-Gait++: Leveraging Spatio-temporal Convolutions for Gait-based Emotion Recognition on Videos

Maria Luísa Lima, Willams De Lima Costa, Estefania Talavera Martínez, Veronica Teichrieb; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 302-310

Emotion recognition is relevant for human behaviour understanding, where facial expression and speech recognition have been widely explored by the computer vision community. Literature in the field of behavioural psychology indicates that gait, described as the way a person walks, is an additional indicator of emotions. In this work, we propose a deep framework for emotion recognition through the analysis of gait. More specifically, our model is composed of a sequence of spatial-temporal Graph Convolutional Networks that produce a robust skeleton-based representation for the task of emotion classification. We evaluate our proposed framework on the E-Gait dataset, composed of a total of 2177 samples. The results obtained represent an improvement of 5% in accuracy compared to the state-of-the-art. In addition, during training we observed a faster convergence of our model compared to the state-of-the-art methodologies.