Soft Label Mining and Average Expression Anchoring for Facial Expression Recognition

Haipeng Ming, Wenhuan Lu, Wei Zhang; Proceedings of the Asian Conference on Computer Vision (ACCV), 2022, pp. 961-977

Facial expression recognition (FER) suffers from high interclass similarity and large intraclass variation, leading to ambiguity or uncertainty and further confusing annotators. They also hinder the network in learning the valuable features of facial expression. Recently, many studies have revealed that the uncertainty or ambiguity is one of the key challenges in FER. In this paper, we propose a new method to address this issue from two aspects: a soft label mining module to convert the original hard labels to soft labels dynamically during training, and an average facial expression anchoring module to separate unique expression features from similarity expression features. The soft label mining module breaks the limits of the categorical model and mitigates the uncertainty or ambiguity. And the average facial expression anchoring module suppresses the high interclass similarity of facial expressions. Our method can train any backbone network for facial expression recognition. The experiments on the popular datasets show that our method achieves state-of-the-art results by 92.82% on RAF-DB and 67.91% on SFEW, and achieves a comparable result of 62.26% on AffectNet. The code is available at https://github.com/HaipengMing/SLM-AEA.