Supervised Contrastive Learning for Generalizable and Explainable DeepFakes Detection

Ying Xu, Kiran Raja, Marius Pedersen; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops, 2022, pp. 379-389

DeepFakes detection approaches have to be agnostic across generation type, quality, and appearance to provide a generalizable DeepFakes detector. Limited generalizability will hinder wide-scale deployment of detectors if they cannot handle unseen attacks in an open set scenario. We propose a generalizable detection model that can detect novel and unknown/unseen DeepFakes using a supervised contrastive (SupCon) loss. As DeepFakes can resemble the original image/video to a greater extent in terms of appearance and it becomes challenging to secern them, we propose to exploit the contrasts in the representation space to learn a generalizable detector. We further investigate the features learnt from our proposed approach for explainability. The analysis for explainability of the models advocates the need for fusion and motivated by this, we fuse the scores from the proposed SupCon model and the Xception network to exploit the variability from different architectures. The proposed model consistently performs better compared to the single model on both known data and unknown attacks consistently in a seen data setting and an unseen data setting, with generalizability and explainability as a basis. We obtain the highest accuracy of 78.74% using proposed SupCon model and an accuracy of 83.99% with proposed fusion in a true open-set evaluation scenario where the test class is unknown at the training phase. The paper also aligns with reproducible research by making the code available.