Late-Resizing: A Simple but Effective Sketch Extraction Strategy for Improving Generalization of Line-Art Colorization

Dohyun Kim, Dajung Je, Kwangjin Lee, Moohyun Kim, Han Kim; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2022, pp. 3469-3478

Automatic line-art colorization is a demanding research field owing to its expensive and labor-intensive workload. Learning-based approaches have lately emerged to improve the quality of colorization. To handle the lack of paired data in line art and color images, sketch extraction has been widely adopted. This study primarily focuses on the resizing process applied within the sketch extraction procedure, which is essential for normalizing input sketches of various sizes to the target size of the colorization model. We first analyze the inherent risk in a conventional resizing strategy, i.e., early-resizing, which places the resizing step before the line detection process to ensure the practicality. Although the strategy is extensively used, it involves an often overlooked risk of significantly degrading the generalization of the colorization model. Thus, we propose a late-resizing strategy in which resizing is applied after the line detection step. The proposed late-resizing strategy has three advantages: prevention of a quality degradation in the color image, augmentation for downsizing artifacts, and alleviation of look-ahead bias. In conclusion, we present both quantitative and qualitative evaluations on representative learning-based line-art colorization methods, which verify the effectiveness of the proposed method in the generalization of the colorization model.