Demographic Differentials in Face Image Quality: Evaluation and Comparison on Real and Synthetic Data

André Dörsch, Johannes Merkle, Benjamin Tams, Gerardo Gutierrez Alvarez, Peter Munch, Christoph Busch, Christian Rathgeb; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2025, pp. 5874-5882

In this work, we investigate potential demographic differentials of quality measures used in the Open Source Face Image Quality (OFIQ) framework. We analyse biometric sample quality of real and synthetic face images across different ethnicities. In addition, we investigate whether the use of synthetic face images is suitable for analysing face image quality assessment algorithms and whether performance trends are consistent between real and synthetic data. Our analysis reveals problematic demographic differentials in the observed quality measures, particularly in the Luminance mean measure with major quality score deviations across ethnic groups. For quality measures that address exposure and brightness patterns, it is particularly noticeable that individuals with darker skin tones tend to have higher discard rates than other individuals, suggesting unfair decision-making. Our findings demonstrate that the overall trend of demographic differentials between synthetic and real identities in our test set is largely similar, with minor exceptions. This underlines both the problematic presence of demographic bias patterns in the observed quality measures across synthetic and real identities and the potential of synthetic data for assessing biases in face image quality assessment algorithms, provided that quality deficiencies are present in appropriate variance.