Spatio-Temporal Vector of Locally Max Pooled Features for Action Recognition in Videos

Ionut Cosmin Duta, Bogdan Ionescu, Kiyoharu Aizawa, Nicu Sebe; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 3097-3106

Abstract


We introduce Spatio-Temporal Vector of Locally Max Pooled Features (ST-VLMPF), a super vector-based encoding method specifically designed for local deep features encoding. The proposed method addresses an important problem of video understanding: how to build a video representation that incorporates the CNN features over the entire video. Feature assignment is carried out at two levels, by using the similarity and spatio-temporal information. For each assignment we build a specific encoding, focused on the nature of deep features, with the goal to capture the highest feature responses from the highest neuron activation of the network. Our ST-VLMPF clearly provides a more reliable video representation than some of the most widely used and powerful encoding approaches (Improved Fisher Vectors and Vector of Locally Aggregated Descriptors), while maintaining a low computational complexity. We conduct experiments on three action recognition datasets: HMDB51, UCF50 and UCF101. Our pipeline obtains state-of-the-art results.

Related Material


[pdf]
[bibtex]
@InProceedings{Duta_2017_CVPR,
author = {Cosmin Duta, Ionut and Ionescu, Bogdan and Aizawa, Kiyoharu and Sebe, Nicu},
title = {Spatio-Temporal Vector of Locally Max Pooled Features for Action Recognition in Videos},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {July},
year = {2017}
}