A Cross-Dataset Study on the Brazilian Sign Language Translation

Amanda Hellen de Avellar Sarmento, Moacir Antonelli Ponti; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 2816-2820

Signed communication is an important form of natural language often less studied, but extremely relevant. The main question we address in this paper is how to translate Brazilian Sign Language implementing Deep Learning networks with limited data availability. Previous studies often use a single dataset, in most cases collected by the authors themselves. We claim a cross-dataset approach would be more adequate to evaluate a real-world scenario. We investigate two methods based on spatial feature extraction. The first one focuses on pre-trained Convolutional Neural Networks and the second one on Body Landmark Estimation (skeleton information). A Long Short-Term Memory network is responsible for the sign classification. Our contribution encompasses data curation, alongside providing general guidelines for enhanced generalization.