Particle Filter Based Probabilistic Forced Alignment for Continuous Gesture Recognition

Necati Cihan Camgoz, Simon Hadfield, Richard Bowden; Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 3079-3085

In this paper, we propose a novel particle filter based probabilistic forced alignment approach for training deep neural networks using weak border level annotations. The proposed method jointly learns to localize and recognize isolated instances in continuous streams. This is done by drawing training volumes from a prior distribution of likely regions and training a discriminative 3D-CNN from this data. The classifier is then used to calculate the posterior distribution by scoring the training examples and using this as the prior for the next sampling stage. We apply the proposed approach to the challenging task of continuous gesture recognition. We evaluate the performance on the popular ChaLearn 2016 ConGD dataset. Our method surpasses state-of-the-art results by obtaining 0.3646 and 0.3744 Mean Jaccard Index Score on the validation and test sets of ConGD, respectively. Furthermore, we participated in the ChaLearn 2017 Continuous Gesture Recognition Challenge and was ranked 3rd.