Can Selfless Learning Improve Accuracy of a Single Classification Task?

Soumya Roy, Bharat Bhusan Sau; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021, pp. 4044-4052

Abstract


The human brain has billions of neurons. However, we perform tasks using only a few concurrently active neurons. Moreover, an activated neuron inhibits the activity of its neighbors. Selfless Learning exploits these neurobiological principles to solve the problem of catastrophic forgetting in continual learning. In this paper, we ask a basic question: can the selfless learning idea be used to improve the accuracy of deep convolutional networks on a single classification task? To achieve this goal, we introduce two regularizers and formulate a curriculum learning-esque strategy to effectively enforce these regularizers on a network. This has resulted in significant gains over vanilla cross-entropy training. Moreover, we have shown that our method can be used in conjunction with other popular learning paradigms like curriculum learning.

Related Material


[pdf]
[bibtex]
@InProceedings{Roy_2021_WACV, author = {Roy, Soumya and Sau, Bharat Bhusan}, title = {Can Selfless Learning Improve Accuracy of a Single Classification Task?}, booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}, month = {January}, year = {2021}, pages = {4044-4052} }