On the Robustness and Generalizability of Face Synthesis Detection Methods

Johan Sabel, Fredrik Johansson; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2021, pp. 962-971

In recent years, significant progress has been made within human face synthesis. It is now possible, and easy for anyone, to generate credible high-resolution images of non-existing people. This calls for effective detection methods. In this paper, three state-of-the-art deep learning-based methods are evaluated with respect to their robustness and generalizability, which are two factors that must be taken into consideration for methods intended to be deployed in the wild. The robustness experiments show that it is possible to achieve near-perfect performance when discriminating between real and synthetic facial images that have been post-processed heavily with various perturbation techniques; especially when similar perturbations are incorporated during training of the detection models. The generalization experiments show that already trained detection models can achieve high performance on images from sources not known during training, provided that the models are fine-tuned on such images. One model achieved an average accuracy of 96.8% after being fine-tuned on 3 training images from each unknown source considered (one real and one synthetic source). However, additional images were required when fine-tuning using a different approach aimed at preventing catastrophic forgetting. Furthermore, in general, no method generalized well without fine-tuning. Hence, the limited generalization capability remains a shortcoming that must be overcome before the detection methods can be utilized in the wild.