Applying Computer Vision to Analyze Self-Injurious Behaviors in Children with Autism Spectrum Disorder

Ayda Eghbalian, Md Mushfiqur Azam, Katie Holloway, Leslie Neely, Kevin Desai; Proceedings of the Winter Conference on Applications of Computer Vision (WACV) Workshops, 2025, pp. 11-20

Computer vision has immense potential to advance healthcare and behavioral research particularly for conditions such as autism spectrum disorder (ASD) which is characterized by early-emerging social communication deficits and repetitive sensory-motor behaviors. Early detection of these behavioral patterns is crucial for timely diagnosis and intervention. While existing studies primarily focus on facial expressions and eye-tracking in children with ASD the study of body gestures remains underexplored largely due to the lack of datasets capturing full-body movements. In this work we present a novel dataset of videos capturing full-body gestures in children with ASD recorded during behavioral therapy sessions. The dataset specifically includes frames capturing severe behaviors such as head hitting and head banging. To establish benchmarks we conducted baseline experiments using state-of-the-art image classification models on these action-specific frames. Furthermore we evaluated kinematic pose estimation models on the same frames analyzing their performance and highlighting the unique challenges faced in applying computer vision techniques to children with ASD. This dataset and our findings aim to bridge critical gaps in current research providing a foundational resource and insights for advancing ASD-focused behavioral assessments through computer vision.