Action Recognition with Temporal Relationships

Guangchun Cheng, Yiwen Wan, Wasana Santiteerakul, Shijun Tang, Bill P. Buckles; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2013, pp. 671-675

Action recognition is an important component in humanmachine interactive systems and video analysis. Besides low-level actions, temporal relationships are also important for many actions, which are not fully studied for recognizing actions. We model the temporal structure of lowlevel actions based on dense trajectory groups. Trajectory groups are a higher level and more meaningful representation of actions than raw individual trajectories. Based on the temporal ordering of trajectory groups, we describe the temporal structure using Allen's temporal relations in a discriminative manner, and combine it with a generative model using bag-of-words. The simple idea behind the model is to extract mid-level features from domain-independent dense trajectories and classify the actions by exploring the temporal structure among them based on a set of Allen's relations. We compare the proposed approach with bag-of-words representation using public datasets, and the results show that our approach improves recognition accuracy.