Pseudo-label Based Unsupervised Fine-tuning of a Monocular 3D Pose Estimation Model for Sports Motions

Tomohiro Suzuki, Ryota Tanaka, Kazuya Takeda, Keisuke Fujii; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 3315-3324

Accurate motion capture is useful for sports motion analysis but requires higher acquisition costs. Monocular or few camera multi-view pose estimation provides an accessible but less accurate alternative especially for sports motion due to training on datasets of daily activities. In addition multi-view estimation is still costly due to camera calibration. Therefore it is desirable to develop an accurate and cost-effective motion capture system for the daily training in sports. In this paper we propose an accurate and convenient sports motion capture system based on unsupervised fine-tuning. The proposed system estimates 3D joint positions by multi-view estimation based on automatic calibration with the human body. These results are used as pseudo-labels for fine-tuning of the recent higher performance monocular 3D pose estimation model. Since the fine-tuning improves the model accuracy for sports motion we can choose multi-view or monocular estimation depending on the situation. We evaluated the system using a running motion dataset and ASPset-510 and showed that fine-tuning improved the performance of monocular estimation to the same level as that of multi-view estimation for running motion. Our proposed system can be useful for the daily motion analysis in sports.