Learning Linear Transformations for Fast Image and Video Style Transfer

Xueting Li, Sifei Liu, Jan Kautz, Ming-Hsuan Yang; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 3809-3817

Abstract


Given a random pair of images, a universal style transfer method extracts the feel from a reference image to synthesize an output based on the look of a content image. Recent algorithms based on second-order statistics, however, are either computationally expensive or prone to generate artifacts due to the trade-off between image quality and runtime performance. In this work, we present an approach for universal style transfer that learns the transformation matrix in a data-driven fashion. Our algorithm is efficient yet flexible to transfer different levels of styles with the same auto-encoder network. It also produces stable video style transfer results due to the preservation of the content affinity. In addition, we propose a linear propagation module to enable a feed-forward network for photo-realistic style transfer. We demonstrate the effectiveness of our approach on three tasks: artistic style, photo-realistic and video style transfer, with comparisons to state-of-the-art methods.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Li_2019_CVPR,
author = {Li, Xueting and Liu, Sifei and Kautz, Jan and Yang, Ming-Hsuan},
title = {Learning Linear Transformations for Fast Image and Video Style Transfer},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}