Exploring Expression-Related Self-Supervised Learning and Spatial Reserve Pooling for Affective Behaviour Analysis

Fanglei Xue, Yifan Sun, Yi Yang; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 5701-5708

Self-supervised learning (SSL) methods have gained attention for reducing dependence on labeled data. However, SSL methods are less investigated for facial expression recognition (FER), which requires expensive expression annotation, especially for large-scale video databases. In this paper, we explore an expression-related self-supervised learning (SSL) method called ContraWarping to perform expression classification in the 5th Affective Behavior Analysis in-the-wild (ABAW) competition. We also conduct a new spatial reserve pooling module to utilize all facial details for expression recognition. By evaluating on the Aff-Wild2 dataset, we demonstrate that ContraWarping outperforms existing supervised methods and other general SSL methods with only 0.7M trainable parameters and shows great application potential in the affective analysis area.