Stylization-Based Architecture for Fast Deep Exemplar Colorization

Zhongyou Xu, Tingting Wang, Faming Fang, Yun Sheng, Guixu Zhang; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 9363-9372

Exemplar-based colorization aims to add colors to a grayscale image guided by a content related reference im- age. Existing methods are either sensitive to the selection of reference images (content, position) or extremely time and resource consuming, which limits their practical applica- tion. To tackle these problems, we propose a deep exemplar colorization architecture inspired by the characteristics of stylization in feature extracting and blending. Our coarse- to-fine architecture consists of two parts: a fast transfer sub-net and a robust colorization sub-net. The transfer sub- net obtains a coarse chrominance map via matching basic feature statistics of the input pairs in a progressive way. The colorization sub-net refines the map to generate the final re- sults. The proposed end-to-end network can jointly learn faithful colorization with a related reference and plausible color prediction with unrelated reference. Extensive exper- imental validation demonstrates that our approach outper- forms the state-of-the-art methods in less time whether in exemplar-based colorization or image stylization tasks.