Joint Patch and Multi-Label Learning for Facial Action Unit Detection

Kaili Zhao, Wen-Sheng Chu, Fernando De la Torre, Jeffrey F. Cohn, Honggang Zhang; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 2207-2216

Abstract


The face is one of the most powerful channel of non-verbal communication. The most commonly used taxonomy to describe facial behaviour is the Facial Action Coding System (FACS). FACS segments the visible effects of facial muscle activation into 30+ action units (AUs). AUs, which may occur alone and in thousands of combinations, can describe nearly all-possible facial expressions. Most existing methods for automatic AU detection treat the problem using one-vs-all classifiers and fail to exploit dependencies among AU and facial features. We introduce joint-patch and multi-label learning (JPML) to address these issues. JPML leverages group sparsity by selecting a sparse subset of facial patches while learning a multi-label classifier. In four of five comparisons on three diverse datasets, CK+, GFT, and BP4D, JPML produced the highest average F1 scores in comparison with state-of-the art.

Related Material


[pdf]
[bibtex]
@InProceedings{Zhao_2015_CVPR,
author = {Zhao, Kaili and Chu, Wen-Sheng and De la Torre, Fernando and Cohn, Jeffrey F. and Zhang, Honggang},
title = {Joint Patch and Multi-Label Learning for Facial Action Unit Detection},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2015}
}