SketchGAN: Joint Sketch Completion and Recognition With Generative Adversarial Network

Fang Liu, Xiaoming Deng, Yu-Kun Lai, Yong-Jin Liu, Cuixia Ma, Hongan Wang; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 5830-5839

Abstract


Hand-drawn sketch recognition is a fundamental problem in computer vision, widely used in sketch-based image and video retrieval, editing, and reorganization. Previous methods often assume that a complete sketch is used as input; however, hand-drawn sketches in common application scenarios are often incomplete, which makes sketch recognition a challenging problem. In this paper, we propose SketchGAN, a new generative adversarial network (GAN) based approach that jointly completes and recognizes a sketch, boosting the performance of both tasks. Specifically, we use a cascade Encode-Decoder network to complete the input sketch in an iterative manner, and employ an auxiliary sketch recognition task to recognize the completed sketch. Experiments on the Sketchy database benchmark demonstrate that our joint learning approach achieves competitive sketch completion and recognition performance compared with the state-of-the-art methods. Further experiments using several sketch-based applications also validate the performance of our method.

Related Material


[pdf]
[bibtex]
@InProceedings{Liu_2019_CVPR,
author = {Liu, Fang and Deng, Xiaoming and Lai, Yu-Kun and Liu, Yong-Jin and Ma, Cuixia and Wang, Hongan},
title = {SketchGAN: Joint Sketch Completion and Recognition With Generative Adversarial Network},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}