Dual-Domain Image Synthesis Using Segmentation-Guided GAN

Dena Bazazian, Andrew Calway, Dima Damen; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022, pp. 507-516


We introduce a segmentation-guided approach to synthesise images that integrate features from two distinct domains. Images synthesised by our dual-domain model belong to one domain within the semantic-mask, and to another in the rest of the image - smoothly integrated. We build on the successes of few-shot StyleGAN and single-shot semantic segmentation to minimise the amount of training required in utilising two domains. The method combines few-shot cross-domain StyleGAN with a latent optimiser to achieve images containing features of two distinct domains. We use a segmentation-guided perceptual loss, which compares both pixel-level and activations between domain-specific and dual-domain synthetic images. Results demonstrate qualitatively and quantitatively that our model is capable of synthesising dual-domain images on a variety of objects (faces, horses, cats, cars), domains (natural, caricature, sketches) and part-based masks (eyes, nose, mouth, hair, car bonnet). Our code is publicly available at: https://github.com/denabazazian/Dual-Domain-Synthesis.

Related Material

[pdf] [supp] [arXiv]
@InProceedings{Bazazian_2022_CVPR, author = {Bazazian, Dena and Calway, Andrew and Damen, Dima}, title = {Dual-Domain Image Synthesis Using Segmentation-Guided GAN}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops}, month = {June}, year = {2022}, pages = {507-516} }