Unsupervised and Semi-Supervised Domain Adaptation for Action Recognition from Drones

Jinwoo Choi, Gaurav Sharma, Manmohan Chandraker, Jia-Bin Huang; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2020, pp. 1717-1726

Abstract


We address the problem of human action classification in drone videos. Due to the high cost of capturing and labeling large-scale drone videos with diverse actions, we present unsupervised and semi-supervised domain adaptation approaches that leverage both the existing fully annotated action recognition datasets and unannotated (or only a few annotated) videos from drones. To study the emerging problem of drone-based action recognition, we create a new dataset, NEC-Drone, containing 5,250 videos to evaluate the task. We tackle both problem settings with 1) same and 2) different action label sets for the source (e.g., Kinectics dataset) and target domains (drone videos). We present a combination of video and instance-based adaptation methods, paired with either a classifier or an embedding-based framework to transfer the knowledge from source to target. Our results show that the proposed adaptation approach substantially improves the performance on these challenging and practical tasks. We further demonstrate the applicability of our method for learning cross-view action recognition on the Charades-Ego dataset. We provide qualitative analysis to understand the behaviors of our approaches.

Related Material


[pdf] [supp] [video]
[bibtex]
@InProceedings{Choi_2020_WACV,
author = {Choi, Jinwoo and Sharma, Gaurav and Chandraker, Manmohan and Huang, Jia-Bin},
title = {Unsupervised and Semi-Supervised Domain Adaptation for Action Recognition from Drones},
booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
month = {March},
year = {2020}
}