Sign Language Recognition: A Large-Scale Multi-View Dataset and Comprehensive Evaluation

Nguyen Son Dinh, Tuan Dung Nguyen, Duc Tri Tran, Nguyen Dang Huy Pham, Thuan Hieu Tran, Ngoc Anh Tong, Quang Huy Hoang, Phi Le Nguyen; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 7876-7886

Vision-based sign language recognition is an extensively researched problem aimed at advancing communication between deaf and hearing individuals. Numerous Sign Language Recognition (SLR) datasets have been introduced to promote research in this field spanning multiple languages vocabulary sizes and signers. However most existing popular datasets focus predominantly on the frontal view of signers neglecting visual information from other perspectives. In practice many sign languages contain words that have similar hand movements and expressions making it challenging to differentiate between them from a single frontal view. Although a few studies have proposed sign language datasets using multi-view data these datasets remain limited in vocabulary size and scale hindering their generalizability and practicality. To address this issue we introduce a new large-scale multi-view sign language recognition dataset spanning 1000 glosses and 30 signers resulting in over 84000 multi-view videos. To the best of our knowledge this is the first multi-view sign language recognition dataset of this scale. In conjunction with offering a comprehensive dataset we perform extensive experiments to assess the performance of state-of-the-art Sign Language Recognition models utilizing on our dataset. The findings indicate that utilizing multi-view data substantially enhances model accuracy across all models with a maximum performance improvement of up to 19.75% compared to models trained on single-view data. Our dataset and baseline models are publicly accessible on GitHub.