SketchBird: Learning To Generate Bird Sketches From Text

Shaozu Yuan, Aijun Dai, Zhiling Yan, Zehua Guo, Ruixue Liu, Meng Chen; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 2443-2452

Sketch plays a critical role in the human art creation process. As one of the functions of the sketch, text-to-sketch may help the artists to catch the fleeting inspirations efficiently. Different from traditional text2image tasks, sketches consist of only a set of sparse lines and depend on very strict edge information, which requires the model to understand the text descriptions accurately and control the shape and texture in the fine-grained granularity. However, there was very rare previous research on the challenging text2sketch task. In this paper, we first construct a text2sketch image dataset by modifying the prevalent CUB dataset. Then a novel Generative Adversarial Network (GAN) based model is proposed by leveraging a Conditional Layer-Instance Normalization (CLIN) module, which can fuse the image features and sentence vector effectively and guide the sketch generation process. Extensive experiments were conducted and the results show the superiority of our proposed model compared to previous baselines. An in-depth analysis was also made to illustrate the contribution of each module and the limitation of our work.