A New Large Scale Dynamic Texture Dataset with Application to ConvNet Understanding

Isma Hadji, Richard P. Wildes; Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 320-335

This paper introduces a new large scale dynamic texture dataset. The dataset is provided with two complementary organizations, one based on dynamics independent of spatial appearance and one based on spatial appearance independent of dynamics. With over 10,000 videos, the proposed Dynamic Texture DataBase (DTDB) is two orders of magnitude larger than any previously available dynamic texture dataset. The complementary organizations of the dataset allow for uniquely insightful experiments regarding the abilities of major classes of spatiotemporal ConvNet architectures to exploit appearance vs. dynamic information. We also present a novel two-stream ConvNet that provides an alterna- tive to the standard optical-flow-based motion stream to broaden the range of dynamic patterns that can be encompassed. The resulting motion stream is shown to outperform the traditional optical flow stream by considerable margins. Finally, the utility of the dataset as a pre-training substrate is demonstrated via transfer learning experiments with a different dynamic texture dataset as well as the companion task of dynamic scene recognition resulting in a new state-of-the-art.