A Semantically Impactful Image Manipulation Dataset: Characterizing Image Manipulations using Semantic Significance

Yuwei Chen, Ming-Ching Chang, Mattias Kirchner, Zhenfei Zhang, Xin Li, Arslan Basharat, Anthony Hoogs; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 7648-7657

We investigate how to characterize semantic significance (SS) in detecting image manipulations (IMD) for media forensics. We introduce the Characterization of Semantic Impact for IMD (CSI-IMD) dataset which focuses on localizing and evaluating the semantic impact of image manipulations to counter advanced generative techniques. Our evaluation of 10 state-of-the-art IMD and localization methods on CSI-IMD reveals key insights. Unlike existing datasets CSI-IMD provides detailed semantic annotations beyond traditional manipulation masks aiding in the development of new defensive strategies. The dataset features manipulations from advanced generation methods offering various levels of semantic significance. It is divided into two parts: a gold-standard set of 1000 manually annotated manipulations with high-quality control and an extended set of 500000 automated manipulations for large-scale training and analysis. We also propose a new SS-focused task to assess the impact of semantically targeted manipulations. Our experiments show that current IMD methods struggle with manipulations created using stable diffusion with TruFor and Cat-Net performing the best among those tested. The CSI-IMD dataset will become available at https://github.com/csiimd/csiimd.