Extreme Low Resolution Action Recognition with Spatial-Temporal Multi-Head Self-Attention and Knowledge Distillation

Didik Purwanto, Rizard Renanda Adhi Pramono, Yie-Tarng Chen, Wen-Hsien Fang; The IEEE International Conference on Computer Vision (ICCV), 2019, pp. 0-0

Abstract


This paper proposes a two-stream network with a novel spatial-temporal multi-head self-attention mechanism for action recognition in extreme low resolution (LR) videos. The new approach first utilizes a super resolution (SR) mechanism to provide better visual information to facilitate the network training. To provide more discriminative spatio-temporal features, a knowledge distillation scheme that consists of teacher and student models is employed to enhance the network model using the knowledge from a high resolution (HR) model. Moreover, the two-stream network is combined with a new spatial-temporal multi-head self-attention network to efficaciously learn the long-term temporal dependency. Simulations demonstrate that the proposed method surpasses the state-of-the-art works for extreme LR action recognition on two widespread HMDB-51 and IXMAS datasets.

Related Material


[pdf]
[bibtex]
@InProceedings{Purwanto_2019_ICCV,
author = {Purwanto, Didik and Renanda Adhi Pramono, Rizard and Chen, Yie-Tarng and Fang, Wen-Hsien},
title = {Extreme Low Resolution Action Recognition with Spatial-Temporal Multi-Head Self-Attention and Knowledge Distillation},
booktitle = {The IEEE International Conference on Computer Vision (ICCV) Workshops},
month = {Oct},
year = {2019}
}