Learning Dynamic GMM for Attention Distribution on Single-Face Videos

Yun Ren, Zulin Wang, Mai Xu, Haoyu Dong, Shengxi Li; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2017, pp. 29-38

The past decade has witnessed the popularity of video conferencing, such as FaceTime and Skype. Hence, it is necessary to predict attention on face videos by saliency detection, as saliency can be used as a guidance of region-of-interest (ROI) for the content-based applications. To this end, this paper proposes a novel approach for saliency detection in single-face videos. Through analysis on our database of 70 single-face videos , we investigate that most attention is attracted by face in videos, and that attention distribution within a face varies with regard to face size and mouth movement. We propose to model visual attention on face region for videos by dynamic Gaussian mixture model (DGMM), the variation of which relies on face size, mouth movement and facial landmarks. Then, we develop a long short-term memory (LSTM) neural network in estimating DGMM for saliency detection of single-face videos, so called LSTM-DGMM.