Multitask Multi-Database Emotion Recognition

Manh Tu Vu, Marie Beurton-Aimar, Serge Marchand; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 3637-3644

This work has been initiated for the 2nd Affective Behavior Analysis in-the-wild (ABAW 2021) competition. We train a unified deep learning model on multi-databases to perform two tasks: seven basic facial expressions prediction and valence-arousal estimation. Since these databases do not contain labels for all the two tasks, we have applied the distillation knowledge technique to train two networks: one teacher and one student model. Both of these models are based on CNN-RNN hybrid architecture. The student model will be trained using both ground truth labels and soft labels derived from the pretrained teacher model. During the training, we have added one more task, which is the combination of the two mentioned tasks, for better exploiting inter-task correlations. We also exploit the sharing videos between the two tasks of the AffWild2 database that is used in the competition for further improving the performance of the network. Experiment results show that with these improvements, our model has reached the performance on par with the state of the art on the test set of the competition. Code and pretrained model are publicly available at https://github.com/glmanhtu/multitask-abaw-2021