Digi2Real: Bridging the Realism Gap in Synthetic Data Face Recognition via Foundation Models

Anjith George, Sebastien Marcel; Proceedings of the Winter Conference on Applications of Computer Vision (WACV) Workshops, 2025, pp. 1469-1478

The accuracy of face recognition systems has improved significantly in the past few years thanks to the large amount of data collected and advancements in neural network architectures. However these large-scale datasets are often collected without explicit consent raising ethical and privacy concerns. To address this there have been proposals to use synthetic datasets for training face recognition models. Yet such models still rely on real data to train the generative models and generally exhibit inferior performance compared to those trained on real datasets. One of these datasets DigiFace uses a graphics pipeline to generate different identities and intra-class variations without using real data in model training. However the performance of this approach is poor on face recognition benchmarks possibly due to the lack of realism in the images generated by the graphics pipeline. In this work we introduce a novel framework for realism transfer aimed at enhancing the realism of synthetically generated face images. Our method leverages the large-scale face foundation model and we adapt the pipeline for realism enhancement. By integrating the controllable aspects of the graphics pipeline with our realism enhancement technique we generate a large amount of realistic variations combining the advantages of both approaches. Our empirical evaluations demonstrate that models trained using our enhanced dataset significantly improve the performance of face recognition systems over the baseline. The source code and dataset will be publicly accessible at the following link: https://www.idiap.ch/paper/digi2real