FaceGest: A Comprehensive Facial Gesture Dataset for Human-Computer Interaction

Yaseen --, Sonain Jamil; Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR) Workshops, 2025, pp. 337-347

Existing benchmarks for facial gesture recognition primarily focus on emotion recognition using static images and a limited number of gesture classes. This limitation constrains the use of facial gestures in human-computer interaction (HCI) and poses challenges in developing real-world applications driven by facial expressions. Unlike traditional emotion recognition datasets, facial expression-driven HCI applications require video-based data to capture dynamic gestures effectively. To address this gap, we introduce FaceGest, a large-scale dynamic facial gesture dataset designed to enhance interaction-based applications. FaceGest includes 13 distinct facial gesture classes encompassing eye-based, mouth-based, head-based, and combined gestures. The dataset was collected under diverse lighting and environmental conditions to ensure robustness. It comprises approximately 15,000 labeled video samples, supplemented with deep feature representations for machine learning research. The dataset creation process involves data collection, augmentation, and structured curation for accessibility. Potential applications of FaceGest span hands-free accessibility solutions, automotive systems, smart home automation, metaverse interactions, security authentication, and gaming controls. By offering this open-access dataset along with baseline models, evaluation metrics, and ablation studies, FaceGest aims to bridge existing gaps in HCI datasets and drive the development of inclusive, efficient, and versatile interaction systems. The dataset, code, and additional resources are publicly available at: https://sonainjameel.github.io/FaceGest/.