Local to Global Learning: Gradually Adding Classes for Training Deep Neural Networks

Hao Cheng, Dongze Lian, Bowen Deng, Shenghua Gao, Tao Tan, Yanlin Geng; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 4748-4756

Abstract


We propose a new learning paradigm, Local to Global Learning (LGL), for Deep Neural Networks (DNNs) to improve the performance of classification problems. The core of LGL is to learn a DNN model from fewer categories (local) to more categories (global) gradually within the entire training set. LGL is most related to the Self-Paced Learning (SPL) algorithm but its formulation is different from SPL. SPL trains its data from simple to complex, while LGL from local to global. In this paper, we incorporate the idea of LGL into the learning objective of DNNs and explain why LGL works better from an information-theoretic perspective. Experiments on the toy data, CIFAR-10, CIFAR-100, and ImageNet dataset show that LGL outperforms the baseline and SPL-based algorithms.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Cheng_2019_CVPR,
author = {Cheng, Hao and Lian, Dongze and Deng, Bowen and Gao, Shenghua and Tan, Tao and Geng, Yanlin},
title = {Local to Global Learning: Gradually Adding Classes for Training Deep Neural Networks},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}