Weakly Supervised Action Localization by Sparse Temporal Pooling Network

Phuc Nguyen, Ting Liu, Gautam Prasad, Bohyung Han; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 6752-6761

Abstract


We propose a weakly supervised temporal action localization algorithm on untrimmed videos using convolutional neural networks. Our algorithm learns from video-level class labels and predicts temporal intervals of human actions with no requirement of temporal localization annotations. We design our network to identify a sparse subset of key segments associated with target actions in a video using an attention module and fuse the key segments through adaptive temporal pooling. Our loss function is comprised of two terms that minimize the video-level action classification error and enforce the sparsity of the segment selection. At inference time, we extract and score temporal proposals using temporal class activations and class-agnostic attentions to estimate the time intervals that correspond to target actions. The proposed algorithm attains state-of-the-art results on the THUMOS14 dataset and outstanding performance on ActivityNet1.3 even with its weak supervision.

Related Material


[pdf] [Supp] [arXiv]
[bibtex]
@InProceedings{Nguyen_2018_CVPR,
author = {Nguyen, Phuc and Liu, Ting and Prasad, Gautam and Han, Bohyung},
title = {Weakly Supervised Action Localization by Sparse Temporal Pooling Network},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2018}
}