A Lightweight Skeleton-Based 3D-CNN for Real-Time Fall Detection and Action Recognition

Nadhira Noor, In Kyu Park; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 2179-2188

Implementing skeleton-based action recognition in real-world applications is a difficult task, because it involves multiple modules such as person detection and pose estimaton. In terms of context, skeleton-based approach has the strong advantage of robustness in understanding actual human actions. However, for most real-world videos in the standard benchmark datasets, human poses are not easy to detect, (i.e. only partially visible or occluded by other objects), and existing pose estimators mostly fail to detect the person during the falling motion. Thus, we propose a newly augmented human pose dataset to improve the accuracy of pose extraction. Furthermore, we propose a lightweight skeleton-based 3D-CNN action recognition network that shows significant improvement on accuracy and processing time over the baseline. Experimental results show that the proposed skeleton-based method shows high accuracy and efficiency in real world scenarios.