Mining Discriminative States of Hands and Objects to Recognize Egocentric Actions With a Wearable RGBD Camera

Shaohua Wan, J.K. Aggarwal; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2015, pp. 36-43

Abstract


Of increasing interest to the computer vision community is to recognize egocentric actions. Conceptually, egocentric actions are largely identifiable by the states of hands and objects. For example, "drinking soda" is essentially composed of two sequential states where one first "opens the bottle", then "drinks from the bottle". While existing algorithms commonly use manually defined states to train action classifiers, we present a novel model that automatically mines discriminative states for recognizing egocentric actions. To mine discriminative states, we propose a novel kernel function and formulate a Multiple Kernel Learning based framework to learn adaptive weights for different states. Our state model demonstrates significant performance improvement over the state-of-the-art methods on 3 benchmark datasets, i.e., RGBD-Ego, ADL, and GTEA.

Related Material


[pdf]
[bibtex]
@InProceedings{Wan_2015_CVPR_Workshops,
author = {Wan, Shaohua and Aggarwal, J.K.},
title = {Mining Discriminative States of Hands and Objects to Recognize Egocentric Actions With a Wearable RGBD Camera},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2015}
}