The Watchlist Imbalance Effect in Biometric Face Identification: Comparing Theoretical Estimates and Empiric Measurements

Pawel Drozdowski, Christian Rathgeb, Christoph Busch; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 3757-3765

Recently, different research groups have found that the gallery composition of a face database can induce performance differentials to facial identification systems in which a probe image is compared against up to all stored reference images to reach a biometric decision. This negative effect has been referred to as ""watchlist imbalance effect by the researchers and exhibits high relevance in real applications of biometrics, most prominently in identification searches against criminal databases and blacklists. In this work, we conduct a detailed analysis of said effect. In particular, we compare empiric observations with theoretical estimates, based on the verification performance across demographic groups and the composition of the used gallery. The experimental evaluations are conducted by systematically varying the size and demographic composition of a cleaned subset of the academic MORPH database and utilising the state-of-the-art open-source ArcFace face recognition system.