Class-Incremental Learning for Action Recognition in Videos

Jaeyoo Park, Minsoo Kang, Bohyung Han; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 13698-13707


We tackle catastrophic forgetting problem in the context of class-incremental learning for video recognition, which has not been explored actively despite the popularity of continual learning. Our framework addresses this challenging task by introducing time-channel importance maps and exploiting the importance maps for learning the representations of incoming examples via knowledge distillation. We also incorporate a regularization scheme in our objective function, which encourages individual features obtained from different time steps in a video to be uncorrelated and eventually improves accuracy by alleviating catastrophic forgetting. We evaluate the proposed approach on brand-new splits of class-incremental action recognition benchmarks constructed upon the UCF101, HMDB51, and Something-Something V2 datasets, and demonstrate the effectiveness of our algorithm in comparison to the existing continual learning methods that are originally designed for image data.

Related Material

[pdf] [supp]
@InProceedings{Park_2021_ICCV, author = {Park, Jaeyoo and Kang, Minsoo and Han, Bohyung}, title = {Class-Incremental Learning for Action Recognition in Videos}, booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)}, month = {October}, year = {2021}, pages = {13698-13707} }