Extracting Local Information from Global Representations for Interpretable Deepfake Detection

Elahe Soltandoost, Richard Plesh, Stephanie Schuckers, Peter Peer, Vitomir Štruc; Proceedings of the Winter Conference on Applications of Computer Vision (WACV) Workshops, 2025, pp. 1629-1639

The detection of deepfakes has become increasingly challenging due to the sophistication of manipulation techniques that produce highly convincing fake videos. Traditional detection methods often lack transparency and provide limited insight into their decision-making processes. To address these challenges we propose in this paper a Locally-Explainable Self-Blended (LESB) DeepFake detector that in addition to the final fake-vs-real classification decision also provides information on which local facial region (i.e. eyes mouth or nose) contributed the most to the decision process. At the heart of the detector is a novel Local Feature Discovery (LFD) technique that can be applied to the embedding space of pretrained DeepFake detectors and allows identifying embedding space directions that encode variations in the appearance of local facial features. We demonstrate the merits of the proposed LFD technique and LESB detector in comprehensive experiments on four popular datasets i.e. Celeb-DF DeepFake Detection Challenge Face Forensics in the Wild and FaceForensics++ and show that the proposed detector is not only competitive in comparison to strong baselines but also exhibits enhanced transparency in the decision-making process by providing insights on the contribution of local face parts in the final detection decision.