Multi-IVE: Privacy Enhancement of Multiple Soft-Biometrics in Face Embeddings

Pietro Melzi, Hatef Otroshi Shahreza, Christian Rathgeb, Ruben Tolosana, Ruben Vera-Rodriguez, Julian Fierrez, S├ębastien Marcel, Christoph Busch; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops, 2023, pp. 323-331


This study focuses on the protection of soft-biometric attributes related to the demographic information of individuals that can be extracted from compact representations of face images, called embeddings. We consider a state-of-the-art technology for soft-biometric privacy enhancement, Incremental Variable Elimination (IVE), and propose Multi-IVE, a new method based on IVE to secure multiple soft-biometric attributes simultaneously. Several aspects of this technology are investigated, proposing different approaches to effectively identify and discard multiple soft-biometric attributes contained in face embeddings. In particular, we consider a domain transformation using Principle Component Analysis (PCA), and apply IVE in the PCA domain. A complete analysis of the proposed Multi-IVE algorithm is carried out studying the embeddings generated by state-of-the-art face feature extractors, predicting soft-biometric attributes contained within them with multiple machine learning classifiers, and providing a cross-database evaluation. The results obtained show the possibility to simultaneously secure multiple soft-biometric attributes and support the application of embedding domain transformations before addressing the enhancement of soft-biometric privacy.

Related Material

@InProceedings{Melzi_2023_WACV, author = {Melzi, Pietro and Shahreza, Hatef Otroshi and Rathgeb, Christian and Tolosana, Ruben and Vera-Rodriguez, Ruben and Fierrez, Julian and Marcel, S\'ebastien and Busch, Christoph}, title = {Multi-IVE: Privacy Enhancement of Multiple Soft-Biometrics in Face Embeddings}, booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops}, month = {January}, year = {2023}, pages = {323-331} }