Demo2Vec: Reasoning Object Affordances From Online Videos

Kuan Fang, Te-Lin Wu, Daniel Yang, Silvio Savarese, Joseph J. Lim; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 2139-2147

Abstract


Watching expert demonstrations is an important way for humans and robots to reason about affordances of unseen objects. In this paper, we consider the problem of reasoning object affordances through the feature embedding of demonstration videos. We design the Demo2Vec model which learns to extract embedded vectors of demonstration videos and predicts the interaction region and the action label on a target image of the same object. We introduce the Online Product Review dataset for Affordance (OPRA) by collecting and labeling diverse YouTube product review videos. Our Demo2Vec model outperforms various recurrent neural network baselines on the collected dataset.

Related Material


[pdf]
[bibtex]
@InProceedings{Fang_2018_CVPR,
author = {Fang, Kuan and Wu, Te-Lin and Yang, Daniel and Savarese, Silvio and Lim, Joseph J.},
title = {Demo2Vec: Reasoning Object Affordances From Online Videos},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2018}
}