Micro-expression recognition using a shallow ConvLSTM-based network

Saurav Shukla, Prabodh Kant Rai, Tanmay T. Verlekar; Proceedings of the Asian Conference on Computer Vision (ACCV) Workshops, 2022, pp. 17-28

Micro-expressions reflect people's genuine emotions, making their recognition of great interest to the research community. Most state-of-the-art methods focus on the use of spatial features to perform micro-expression recognition. Thus, they fail to capture the spatiotemporal information available in a video sequence. This paper proposes a shallow convolutional long short-term memory (ConvLSTM) based network to perform micro-expression recognition. The convolutional and recurrent structures within the ConvLSTM allow the network to effectively capture the spatiotemporal information available in a video sequence. To highlight its effectiveness, the proposed ConvLSTM-based network is evaluated on the SAMM dataset. It is trained to perform micro-expression recognition across three (positive, negative, and surprise) and five (happiness, other, anger, contempt, and surprise) classes. When compared with the state-of-the-art, the results report a significant improvement in accuracy and the F1 score. The proposal is also robust against the unbalanced class sizes of the SAMM dataset.