Posture-Based Infant Action Recognition in the Wild With Very Limited Data

Xiaofei Huang, Lingfei Luan, Elaheh Hatamimajoumerd, Michael Wan, Pooria Daneshvar Kakhaki, Rita Obeid, Sarah Ostadabbas; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 4912-4921

Automatic detection of infant actions from home videos could aid medical and behavioral specialists in the early detection of motor impairments in infancy. However, most computer vision approaches for action recognition are centered around adult subjects, following datasets and benchmarks in the field. In this work, we present a data-efficient pipeline for infant action recognition based on the idea of modeling an action as a time sequence consisting of two different stable postures with a transition period between them. The postures are detected frame-wise from the estimated 2D and 3D infant body poses and the action sequence is segmented based on the posture-driven low-dimensional features of each frame. To spur further research in the field, we also created and release the first-of-its-kind infant action dataset---InfAct---consisting of 200 fully annotated home videos representing a wide range of common infant actions, intended as a public benchmark. Among the ten more common classes of infant actions, our action recognition model achieved 78.0% accuracy when tested on InfAct, highlighting the promise of video-based infant action recognition as a viable monitoring tool for infant motor development.