Style Transfer by Relaxed Optimal Transport and Self-Similarity

Nicholas Kolkin, Jason Salavon, Gregory Shakhnarovich; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 10051-10060

Abstract


The goal of style transfer algorithms is to render the content of one image using the style of another. We propose Style Transfer by Relaxed Optimal Transport and Self-Similarity (STROTSS), a new optimization-based style transfer algorithm. We extend our method to allow user specified point-to-point or region-to-region control over visual similarity between the style image and the output. Such guidance can be used to either achieve a particular visual effect or correct errors made by unconstrained style transfer. In order to quantitatively compare our method to prior work, we conduct a large-scale user study designed to assess the style-content tradeoff across settings in style transfer algorithms. Our results indicate that for any desired level of content preservation, our method provides higher quality stylization than prior work.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Kolkin_2019_CVPR,
author = {Kolkin, Nicholas and Salavon, Jason and Shakhnarovich, Gregory},
title = {Style Transfer by Relaxed Optimal Transport and Self-Similarity},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}