MMFace: A Multi-Metric Regression Network for Unconstrained Face Reconstruction

Hongwei Yi, Chen Li, Qiong Cao, Xiaoyong Shen, Sheng Li, Guoping Wang, Yu-Wing Tai; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 7663-7672

Abstract


We propose to address the face reconstruction in the wild by using a multi-metric regression network, MMFace, to align a 3D face morphable model (3DMM) to an input image. The key idea is to utilize a volumetric sub-network to estimate an intermediate geometry representation, and a parametric sub-network to regress the 3DMM parameters. Our parametric sub-network consists of identity loss, expression loss, and pose loss which greatly improves the aligned geometry details by incorporating high level loss functions directly defined in the 3DMM parametric spaces. Our high-quality reconstruction is robust under large variations of expressions, poses, illumination conditions, and even with large partial occlusions. We evaluate our method by comparing the performance with state-of-the-art approaches on latest 3D face dataset LS3D-W and Florence. We achieve significant improvements both quantitatively and qualitatively. Due to our high-quality reconstruction, our method can be easily extended to generate high-quality geometry sequences for video inputs.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Yi_2019_CVPR,
author = {Yi, Hongwei and Li, Chen and Cao, Qiong and Shen, Xiaoyong and Li, Sheng and Wang, Guoping and Tai, Yu-Wing},
title = {MMFace: A Multi-Metric Regression Network for Unconstrained Face Reconstruction},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}