TransGaGa: Geometry-Aware Unsupervised Image-To-Image Translation

Wayne Wu, Kaidi Cao, Cheng Li, Chen Qian, Chen Change Loy; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 8012-8021

Abstract


Unsupervised image-to-image translation aims at learning a mapping between two visual domains. However, learning a translation across large geometry variations al- ways ends up with failure. In this work, we present a novel disentangle-and-translate framework to tackle the complex objects image-to-image translation task. Instead of learning the mapping on the image space directly, we disentangle image space into a Cartesian product of the appearance and the geometry latent spaces. Specifically, we first in- troduce a geometry prior loss and a conditional VAE loss to encourage the network to learn independent but com- plementary representations. The translation is then built on appearance and geometry space separately. Extensive experiments demonstrate the superior performance of our method to other state-of-the-art approaches, especially in the challenging near-rigid and non-rigid objects translation tasks. In addition, by taking different exemplars as the ap- pearance references, our method also supports multimodal translation. Project page: https://wywu.github. io/projects/TGaGa/TGaGa.html

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Wu_2019_CVPR,
author = {Wu, Wayne and Cao, Kaidi and Li, Cheng and Qian, Chen and Loy, Chen Change},
title = {TransGaGa: Geometry-Aware Unsupervised Image-To-Image Translation},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}