Rethinking Visibility in Human Pose Estimation: Occluded Pose Reasoning via Transformers

Pengzhan Sun, Kerui Gu, Yunsong Wang, Linlin Yang, Angela Yao; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024, pp. 5903-5912

Occlusion is a common challenge in human pose estimation. Curiously, learning from occluded keypoints hinders a model to detect visible keypoints. We speculate that the impairment is likely due to a forced correlation between keypoints and visual features of the occluders. As such, we propose a novel visibility-aware attention mechanism to eliminate unreliable occluding features. The explicit occlusion handling encourages the model to reason about occluded keypoints using evidence and contextual information from the visible keypoints. It also mitigates the damage of unreliable correlations of the occluded keypoints. Our method, when added to the strong baseline SimCC, improves by 1.3 AP and 0.7 AP with ResNet and HRNet respectively. It also surpasses the state-of-the-art I^2R-Net on CrowdPose by 0.3 AP and 0.6 AP^hard. The improvements highlight that rethinking visibility information is critical for developing effective human pose estimation systems.