Exp-GAN: 3D-Aware Facial Image Generation with Expression Control

Yeonkyeong Lee, Taeho Choi, Hyunsung Go, Hyunjoon Lee, Sunghyun Cho, Junho Kim; Proceedings of the Asian Conference on Computer Vision (ACCV), 2022, pp. 3812-3827


This paper introduces Exp-GAN, a 3D-aware facial image generator with explicit control of facial expressions. Unlike previous 3D-aware GANs, Exp-GAN supports fine-grained control over facial shapes and expressions disentangled from poses. To this ends, we propose a novel hybrid approach that adopts a 3D morphable model (3DMM) with neural textures for the facial region and a neural radiance field (NeRF) for non-facial regions with multi-view consistency. The 3DMM allows fine-grained control over facial expressions, whereas the NeRF contains volumetric features for the non-facial regions. The two features, generated separately, are combined seamlessly with our depth-based integration method that integrates the two complementary features through volume rendering. We also propose a training scheme that encourages generated images to reflect control over shapes and expressions faithfully. Experimental results show that the proposed approach successfully synthesizes realistic view consistent face images with fine-grained controls. Code is available at https://github.com/kakaobrain/expgan.

Related Material

[pdf] [supp] [code]
@InProceedings{Lee_2022_ACCV, author = {Lee, Yeonkyeong and Choi, Taeho and Go, Hyunsung and Lee, Hyunjoon and Cho, Sunghyun and Kim, Junho}, title = {Exp-GAN: 3D-Aware Facial Image Generation with Expression Control}, booktitle = {Proceedings of the Asian Conference on Computer Vision (ACCV)}, month = {December}, year = {2022}, pages = {3812-3827} }