Self-Supervised 3D Hand Pose Estimation Through Training by Fitting

Chengde Wan, Thomas Probst, Luc Van Gool, Angela Yao; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 10853-10862

Abstract


We present a self-supervision method for 3D hand pose estimation from depth maps. We begin with a neural network initialized with synthesized data and fine-tune it on real but unlabelled depth maps by minimizing a set of data-fitting terms. By approximating the hand surface with a set of spheres, we design a differentiable hand renderer to align estimates by comparing the rendered and input depth maps. In addition, we place a set of priors including a data-driven term to further regulate the estimate's kinematic feasibility. Our method makes highly accurate estimates comparable to current supervised methods which require large amounts of labelled training samples, thereby advancing state-of-the-art in unsupervised learning for hand pose estimation.

Related Material


[pdf]
[bibtex]
@InProceedings{Wan_2019_CVPR,
author = {Wan, Chengde and Probst, Thomas and Gool, Luc Van and Yao, Angela},
title = {Self-Supervised 3D Hand Pose Estimation Through Training by Fitting},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}