Beyond the Screen: Evaluating Deepfake Detectors under Moire Pattern Effects

Razaib Tariq, Minji Heo, Simon S. Woo, Shahroz Tariq; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 4429-4439

The detection of deepfakes is crucial for mitigating the societal impact of falsified video content. Despite the development of various algorithms for this purpose challenges arise for detectors in real-world scenarios especially when users capture deepfake content from screens and upload it online or when detectors operate on external devices like smartphones requiring the capture of potential deepfakes through the camera for evaluation. A significant challenge in these scenarios is the presence of Moire patterns which degrade image quality and complicate conventional classification methods notably deep neural networks (DNNs). However the impact of Moire patterns on the effectiveness of deepfake detection systems has not been adequately explored. This study aims to investigate how capturing deepfake videos via digital screen cameras affects the accuracy of detection mechanisms. We introduced the Moire patterns by capturing the display of a monitor using a smartphone camera and conducted empirical evaluations using four widely recognized datasets: CelebDF DFD DFDC and FF++. We compare the performance of twelve SOTA detectors on deepfake videos captured under the influence of Moire patterns. Our findings reveal a performance decrease of up to 33.1 and 31.3 percentage points for image- and video-based detectors. Therefore highlighting the challenges posed by Moire patterns and other naturally induced artifacts is critical for improving the effectiveness of real-world deepfake detection efforts. To facilitate further research we will release the Moire pattern impact version of CelebDF DFD DFDC and FF++ datasets with this paper. Our code is available here: https://github.com/Razaib-Tariq/deepmoire