Quality-based Artifact Modeling for Facial Deepfake Detection in Videos

Sara Concas, Simone Maurizio La Cava, Roberto Casula, Giulia Orrù, Giovanni Puglisi, Gian Luca Marcialis; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 3845-3854

Facial deepfakes are becoming more and more realistic to the point that it is often difficult for humans to distinguish between a fake and a real video. However it is acknowl- edged that deepfakes contain artifacts at different levels; we hypothesize a connection between manipulations and visi-ble or non-visible artifacts especially where the subject's movements are difficult to reproduce in detail. Accordingly our approach relies on different quality measures No-Reference (NR) and Full-Reference (FR) over the detected faces in the video. The measurements allow us to adopt a frame-by-frame approach to build an effective matrix-based representation of a video sequence. We show that the results obtained by this basic feature set for a neural network architecture constitute the first step that encourages the empowerment of this representation aimed to extend our investigation to further deepfake classes. The FaceForensics++ dataset is chosen for experiments which allows the evaluation of the proposed approach over different deepfake generation algorithms.