Learning to Select Pre-Trained Deep Representations With Bayesian Evidence Framework

Yong-Deok Kim, Taewoong Jang, Bohyung Han, Seungjin Choi; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 5318-5326

Abstract


We propose a Bayesian evidence framework to facilitate transfer learning from pre-trained deep convolutional neural networks (CNNs). Our framework is formulated on top of a least squares SVM (LS-SVM) classifier, which is simple and fast in both training and testing, and achieves competitive performance in practice. The regularization parameters in LS-SVM is estimated automatically without grid search and cross-validation by maximizing evidence, which is a useful measure to select the best performing CNN out of multiple candidates for transfer learning; the evidence is optimized efficiently by employing Aitken's delta-squared process, which accelerates convergence of fixed point update. The proposed Bayesian evidence framework also provides a good solution to identify the best ensemble of heterogeneous CNNs through a greedy algorithm. Our Bayesian evidence framework for transfer learning is tested on 12 visual recognition datasets and illustrates the state-of-the-art performance consistently in terms of prediction accuracy and modeling efficiency.

Related Material


[pdf] [video]
[bibtex]
@InProceedings{Kim_2016_CVPR,
author = {Kim, Yong-Deok and Jang, Taewoong and Han, Bohyung and Choi, Seungjin},
title = {Learning to Select Pre-Trained Deep Representations With Bayesian Evidence Framework},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2016}
}