Detailed Human Shape Estimation From a Single Image by Hierarchical Mesh Deformation

Hao Zhu, Xinxin Zuo, Sen Wang, Xun Cao, Ruigang Yang; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 4491-4500

Abstract


This paper presents a novel framework to recover detailed human body shapes from a single image. It is a challenging task due to factors such as variations in human shapes, body poses, and viewpoints. Prior methods typically attempt to recover the human body shape using a parametric based template that lacks the surface details. As such the resulting body shape appears to be without clothing. In this paper, we propose a novel learning-based framework that combines the robustness of parametric model with the flexibility of free-form 3D deformation. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation (HMD) framework, utilizing the constraints from body joints, silhouettes, and per-pixel shading information. We are able to restore detailed human body shapes beyond skinned models. Experiments demonstrate that our method has outperformed previous state-of-the-art approaches, achieving better accuracy in terms of both 2D IoU number and 3D metric distance. The code is available in https://github.com/zhuhao-nju/hmd.git.

Related Material


[pdf]
[bibtex]
@InProceedings{Zhu_2019_CVPR,
author = {Zhu, Hao and Zuo, Xinxin and Wang, Sen and Cao, Xun and Yang, Ruigang},
title = {Detailed Human Shape Estimation From a Single Image by Hierarchical Mesh Deformation},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}