Facial Hair Area in Face Recognition Across Demographics: Small Size, Big Effect

Haiyu Wu, Sicong Tian, Aman Bhatta, Kağan Öztürk, Karl Ricanek, Kevin W. Bowyer; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops, 2024, pp. 1131-1140

Observed variations in face recognition accuracy across demographics, often viewed as "bias", have motivated research into the causes of such variations. Variations in facial hairstyle are an important potential cause of accuracy differences for males. In this work, we first explore how face recognition accuracy is affected by the facial hair region - clean-shaven, mustache, chin-area beard, side-to-side beard. Results show that mustache area facial hair has a greater effect on accuracy than either chin-area beard or side-to-side beard. We then employ a synthetic facial hair method to verify the consistency of the observation across five pixel distributions and three face matchers. Results of these experiments indicate that, the larger the difference in pixel distribution between facial hair region and skin region, the larger impact of the mustache area. To reduce accuracy differences caused by facial hairstyle, quantified by Dd', we adjust the training dataset distribution to have increased representation of facial hair, resulting in an over 40% reduction in accuracy difference.