Detection of Metadata Tampering Through Discrepancy Between Image Content and Metadata Using Multi-Task Deep Learning

Bor-Chun Chen; Pallabi Ghosh; Vlad I. Morariu; Larry S. Davis; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2017, pp. 60-68

Image content or metadata editing software availability and ease of use has resulted in a high demand for automatic image tamper detection algorithms. Most previous work has focused on detection of tampered image content, whereas we develop techniques to detect metadata tampering in outdoor images using sun altitude angle and other meteorological information like temperature, humidity and weather, which can be observed in most outdoor image scenes. To train and evaluate our technique, we create a large dataset of outdoor images labeled with sun altitude angle and other meteorological data (AMOS+M2), which to our knowledge, is the largest publicly available dataset of its kind. Using this dataset, we train separate regression models for sun altitude angle, temperature and humidity and a classification model for weather to detect any discrepancy between image content and its metadata. Finally, a joint multi-task network for these four features shows a relative improvement of 15.5% compared to each of them individually. We include a detailed analysis for using these networks to detect various types of modification to location and time information in image metadata.