Deep Texture and Structure Aware Filtering Network for Image Smoothing

Kaiyue Lu, Shaodi You, Nick Barnes; Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 217-233

Image smoothing is a fundamental task in computer vision, which aims to retain salient structures and remove insignificant textures. In this paper, we tackle the natural deficiency of existing methods, that they cannot properly distinguish textures and structures with similar low-level appearance. While deep learning approaches have addressed preserving structures, they do not yet properly address textures. To this end, we generate a large dataset by blending natural textures with clean structure-only images, and then build a texture prediction network (TPN) that predicts location and magnitude of textures. After that, we combine it with a semantic structure prediction network (SPN) so that the final texture and structure aware filtering network (TSAFN) is informed the textures to remove ("texture-awareness") and the structures to preserve ("structure-awareness"). The proposed model is easy to understand and implement, and shows excellent performance on real images in the wild as well as our generated dataset.