OSCAR-Net: Object-Centric Scene Graph Attention for Image Attribution

Eric Nguyen, Tu Bui, Viswanathan Swaminathan, John Collomosse; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 14499-14508

Abstract


Images tell powerful stories but cannot always be trusted. Matching images back to trusted sources (attribution) enables users to make a more informed judgment of the images they encounter online. We propose a robust image hashing algorithm to perform such matching. Our hash is sensitive to manipulation of subtle, salient visual details that can substantially change the story told by an image. Yet the hash is invariant to benign transformations (changes in quality, codecs, sizes, shapes, etc.) experienced by images during online redistribution. Our key contribution is OSCAR-Net (Object-centric Scene Graph Attention for Image Attribution Network); a robust image hashing model inspired by recent successes of Transformers in the visual domain. OSCAR-Net constructs a scene graph representation that attends to fine-grained changes of every object's visual appearance and their spatial relationships. The network is trained via contrastive learning on a dataset of original and manipulated images yielding a state of the art image hash for content fingerprinting that scales to millions of images.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Nguyen_2021_ICCV, author = {Nguyen, Eric and Bui, Tu and Swaminathan, Viswanathan and Collomosse, John}, title = {OSCAR-Net: Object-Centric Scene Graph Attention for Image Attribution}, booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)}, month = {October}, year = {2021}, pages = {14499-14508} }