Dependency-aware Attention Control for Unconstrained Face Recognition with Image Sets

Xiaofeng Liu, B.V.K Vijaya Kumar, Chao Yang, Qingming Tang, Jane You; Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 548-565

This paper targets the problem of image set-based face verification and identification. Unlike traditional single media (an image or video) setting, we encounter a set of heterogeneous contents containing orderless images and videos. The importance of each image is usually considered either equal or based on their independent quality assessment. How to model the relationship of orderless images within a set remains a challenge. We address this problem by formulating it as a Markov Decision Process (MDP) at the feature level. Specifically, we propose a dependency-aware attention control (DAC) network, which resorts to actor-critic reinforcement learning for sequential attention decision of each image embedding to fully exploit the rich correlation cues among the unordered images. Moreover, its sample-efficient variant with off-policy experience replay is introduced to speed up the learning process. The pose-guided representation scheme can further boost the performance at the extremes of the pose variation. We show that our method leads to the state-of-the-art accuracy on IJB-A dataset and also generalizes well in several video-based face recognition tasks, extit{e.g.}, YTF and Celebrity-1000.