H^3Net: Irregular Posture Detection by Understanding Human Character and Core Structures

Seungha Noh, Kangmin Bae, Yuseok Bae, Byong-Dai Lee; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 5631-5641

This paper proposes H^3Net that considers detecting people in irregular postures by utilizing human structures and characters. To handle both features we introduce two attention modules: 1) Human Structure Attention Module (HSAM) which is introduced to focus on the spatial aspects of a person and 2) Human Character Attention Module (HCAM) which is designed to address the issue of repetitive appearance. HSAM effectively handles both foreground and background information of a human instance and utilizes keypoints to provide additional guidance to predict irregular postures. Meanwhile HCAM employs ID information obtained from the tracking head enriching the posture prediction with high-level semantic information. Furthermore gathering images of people in irregular postures is a challenging task. Therefore many conventional datasets consist of images with the same actors simulating varying postures in distinct images. To address this problem we propose a Human ID Dependent Posture (HID^2) loss which handles repeated instances. The HID^2 loss generates a regularization term by considering duplicated instances to reduce bias. Our experiments demonstrate the effectiveness of H^3Net compared to existing algorithms on irregular posture datasets. Furthermore we show the qualitative results using color-coded masks and bounding boxes. We also provide ablation studies to highlight the significance of our proposed methods.