From Scarcity to Understanding: Transfer Learning for the Extremely Low Resource Irish Sign Language

Ruth Holmes, Ellen Rushe, Mathieu De Coster, Maxim Bonnaerens, Shin'ichi Satoh, Akihiro Sugimoto, Anthony Ventresque; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 2008-2017

One of the most significant challenges to sign language recognition (SLR) today is the low resource nature of sign language datasets, with many datasets being extremely low resource. Transfer learning is therefore a promising, and likely indispensable, method of increasing recognition performance. The use of pose estimation models, which are typically trained on a large and diverse population, can also aid generalization for extremely low resource sign languages. However, research on transfer learning for pose estimation keypoints as inputs has been limited. In this work, we explore transfer learning as a means to improve SLR classification performance for the extremely low resource Irish Sign Language (ISL). We show that transfer learning on larger datasets containing secondary sign languages significantly improves performance on our target sign language, ISL. To understand these results and the attributes that make one dataset better than another for pre-training, we analyse the linguistic relationships between these datasets. We find that certain attributes of datasets are associated with better transfer learning performance. We hope that our findings will not only motivate further research into transfer learning for pose keypoint-based SLR but also act as a practical guide to researchers on choosing the most suitable datasets with which to pre-train models.