Enhancing Soft Biometric Face Template Privacy With Mutual Information-Based Image Attacks

Zohra Rezgui, Nicola Strisciuglio, Raymond Veldhuis; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops, 2024, pp. 1141-1149

The features learned by deep-learning based face recognition networks pose privacy risks as they encode sensitive information that could be used to infer demographic attributes. In this paper, we propose an image-based solution that enhances the soft biometric privacy of the templates generated by face recognition networks. The method uses a reliable mutual information estimation and simulates a minimization step of the mutual information between the features and the target variable. We comprehensively assess the effectiveness of our approach on the gender classification task by formulating two distinct evaluation settings: one for evaluating the performance of the approach's ability to fool a given gender classifier and another for evaluating its ability to hinder the separability of the gender distributions. We conduct an extensive analysis, considering varying levels of perturbation. We show the potential of our method as a privacy-enhancing method that preserves the verification performance as well as a strong single-step adversarial attack.