Information Elevation Network for Online Action Detection and Anticipation

Sunah Min, Jinyoung Moon; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022, pp. 2550-2558

Given a partially observed video segment, online action detection and anticipation aim to identify a current action and forecast future actions, respectively. To detect actions in a streaming video for monitoring applications including surveillance and autonomous driving, online action detection methods have been proposed. Considering the importance of current action in online action detection, we introduce a novel information elevation unit (IEU) that lifts and accumulates the past information relevant to the current action, to compensate for forgotten essential information. Using the IEUs, we propose an information elevation network (IEN) that effectively identifies a current action and anticipates future actions through the dense prediction of past and current action classes within the video segment. For its practical use in online monitoring applications, our IEN takes visual features extracted from a fast action recognition using only RGB frames because extracting optical flows requires heavy computation overhead. On THUMOS-14 and TVSeries, our IEN outperforms state-of-the-art methods using only RGB frames. Furthermore, on the THUMOS-14 dataset, our IEN outperforms the state-of-the-art methods.