Deepfake Catcher: Can a Simple Fusion be Effective and Outperform Complex DNNs?

Akshay Agarwal, Nalini Ratha; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 3791-3801

Despite having completely different configurations deep learning architectures learn a specific set of features that are common across architectures. For example the initial few layers learn the low-level edge features from the images. Based on this fact in this research we have showcased the potential of deep neural network fusion for simple and effective deepfake detection. The advantage of building an architecture in such a manner is to build a low-power-consuming and accurate defense that can be deployed on mobile devices. To utilize the pre-trained knowledge and obtain downstream task-specific knowledge we have identified a breakpoint in different networks and divided the obtained knowledge of a network into fixed and adaptive information. We have kept the fixed knowledge intact while modifying the adaptive knowledge along with entirely new knowledge for the deepfake detection task. In the end the decision of multiple deep architectures trained based on their breakpoint are combined for improved performance. Extensive comparisons performed with existing state-of-the-art architectures demonstrate the effectiveness of the proposed deepfake detection algorithm. The proposed algorithm not only surpasses the existing state-of-the-art (SOTA) algorithms but also needs low computational power. We have further challenged the proposed algorithm by evaluating it by collecting real-world deepfake images.