Privacy-Preserving Annotation of Face Images Through Attribute-Preserving Face Synthesis

Sola Shirai, Jacob Whitehill; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2019, pp. 0-0

We investigate the viability of collecting annotations for face images while preserving privacy by using synthesized images as surrogates. We compare two approaches: a state-of-the-art 3-D face model based on deep neural networks (Extreme3D) to render a detailed 3-D reconstruction of the face from an input image; and a novel generative adversarial network architecture that we propose that extends BEGAN-CS to generate images conditioned on desired low-level facial attributes. Using these two alternative models, we conduct experiments on Mechanical Turk to annotate emotions ("joy" and "anger") on raw and synthesized versions of face images. Across 60 workers each annotating 3 versions of 60 images in each experiment, we find that: (1) The labeling accuracy when viewing surrogate images can be very similar to the accuracy when viewing raw images, but depends significantly on the labeling task. (2) The proposed extension to BEGAN-CS is effective in generating realistic images that correspond to the input vector of low-level facial attributes. (3) Overall, the GAN-based approach to generating surrogate images gives comparable accuracy as the 3-D face model, but is easier to train.