Confidence-Aware RGB-D Face Recognition via Virtual Depth Synthesis

Zijian Chen, Mei Wang, Weihong Deng, Hongzhi Shi, Dongchao Wen, Yingjie Zhang, Xingchen Cui, Jian Zhao; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 1481-1489

2D face recognition encounters challenges in unconstrained environments due to varying illumination occlusion and pose. Recent studies focus on RGB-D face recognition to improve robustness by incorporating depth information. However collecting sufficient paired RGB-D training data is expensive and time-consuming hindering wide deployment. In this work we first construct a diverse depth dataset generated by 3D Morphable Models for depth model pre-training. Then we propose a domain-independent pre-training framework that utilizes readily available pre-trained RGB and depth models to separately perform face recognition without needing additional paired data for retraining. To seamlessly integrate the two distinct networks and harness the complementary benefits of RGB and depth information for improved accuracy we propose an innovative Adaptive Confidence Weighting (ACW). This mechanism is designed to learn confidence estimates for each modality to achieve modality fusion at the score level. Our method is simple and lightweight only requiring ACW training beyond the backbone models. Experiments on multiple public RGB-D face recognition benchmarks demonstrate state-of-the-art performance surpassing previous methods based on depth estimation and feature fusion validating the efficacy of our approach.