RAST: Restorable Arbitrary Style Transfer via Multi-Restoration

Yingnan Ma, Chenqiu Zhao, Xudong Li, Anup Basu; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023, pp. 331-340

Arbitrary style transfer aims at reproducing the target image with provided artistic or photo-realistic styles. Even though existing approaches can successfully transfer style information, arbitrary style transfer still faces many challenges, such as the content leak issue. To be specific, the embedding of artistic style can lead to content changes. In this paper, we solve the content leak problem from the perspective of image restoration. In particular, an iterative architecture is proposed to achieve the restorable arbitrary style transfer (RAST), which can realize the transmission of both content and style information through the multi-restorations. We control the content-style balance in stylized images by the accuracy of image restoration. In order to ensure the effectiveness of the proposed RAST architecture, we design two novel loss functions: multi-restoration loss and style difference loss. In addition, we propose a new quantitative evaluation method to measure content preservation performance and style embedding performance. Comprehensive experiments comparing with state-of-the-art methods demonstrate that our proposed architecture can produce stylized images with superior performance on content preservation and style embedding.