Overcoming the Trade-Off Between Accuracy and Plausibility in 3D Hand Shape Reconstruction

Ziwei Yu, Chen Li, Linlin Yang, Xiaoxu Zheng, Michael Bi Mi, Gim Hee Lee, Angela Yao; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 544-553

Abstract

Direct mesh fitting for 3D hand shape reconstruction estimates highly accurate meshes. However, the resulting meshes are prone to artifacts and do not appear as plausible hand shapes. Conversely, parametric models like MANO ensure plausible hand shapes but are not as accurate as the non-parametric methods. In this work, we introduce a novel weakly-supervised hand shape estimation framework that integrates non-parametric mesh fitting with MANO models in an end-to-end fashion. Our joint model overcomes the tradeoff in accuracy and plausibility to yield well-aligned and high-quality 3D meshes, especially in challenging two-hand and hand-object interaction scenarios.

Related Material

[pdf] [supp] [arXiv]
[bibtex]
@InProceedings{Yu_2023_CVPR, author = {Yu, Ziwei and Li, Chen and Yang, Linlin and Zheng, Xiaoxu and Mi, Michael Bi and Lee, Gim Hee and Yao, Angela}, title = {Overcoming the Trade-Off Between Accuracy and Plausibility in 3D Hand Shape Reconstruction}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2023}, pages = {544-553} }