Hand Gesture Based Region Marking for Tele-Support Using Wearables

Archie Gupta, Shreyash Mohatta, Jitender Maurya, Ramakrishna Perla, Ramya Hebbalaguppe, Ehtesham Hassan; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2017, pp. 69-75

Wearable Augmented Reality devices are being explored in many applications for visualizing real-time contextual information. More importantly, these devices can also be used in tele-assistance from remote sites when on-field operators require off-field expert's guidance for trouble-shooting. For an effective communication, touchless hand gestures are the most intuitive to select a Region Of Interest (ROI) like defective parts in a machine, through a wearable. This paper presents a hand gestural interaction method to localise the ROI in FPV. Novelty of the proposed method include (a)touchless finger based gesture recognition algorithm that runs on smartphones, which can be used with wearable frugal modality like Google Cardboard/Wearality, (b)reducing the network latency and achieving real-time performance by on-board implementation of recognition module. We also conducted user studies that suggest the usefulness of the proposed method and evaluated it using the PASCAL VOC criteria.