Concurrence-Aware Long Short-Term Sub-Memories for Person-Person Action Recognition

Xiangbo Shu, Jinhui Tang, Guo-Jun Qi, Yan Song, Zechao Li, Liyan Zhang; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2017, pp. 1-8

Abstract


Recently, Long Short-Term Memory (LSTM) has become a popular choice to model individual dynamics for single-person action recognition. However, existing RNN models only focus on capturing the temporal dynamics of the person-person interactions by naively combining the activity dynamics of individuals or modeling them as a whole. This neglects the inter-related dynamics of how person-person interactions change over time. To this end, we propose a novel Concurrent Long Short-Term Memories (Co-LSTM) to model the long-term inter-related dynamics between two interacting people on the bonding boxes covering people. Specifically, for each frame, two sub-memory units store individual motion information, while a concurrent LSTM unit selectively integrates and stores inter-related motion information between interacting people from these two sub-memory units via a new co-memory cell. In experiments, we show the superior performance of Co-LSTM compared with the state-of-the-arts.

Related Material


[pdf] [arXiv]
[bibtex]
@InProceedings{Shu_2017_CVPR_Workshops,
author = {Shu, Xiangbo and Tang, Jinhui and Qi, Guo-Jun and Song, Yan and Li, Zechao and Zhang, Liyan},
title = {Concurrence-Aware Long Short-Term Sub-Memories for Person-Person Action Recognition},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {July},
year = {2017}
}