A Main/Subsidiary Network Framework for Simplifying Binary Neural Networks

Yinghao Xu, Xin Dong, Yudian Li, Hao Su; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 7154-7162

Abstract


To reduce memory footprint and run-time latency, techniques such as neural net-work pruning and binarization have been explored separately. However, it is un-clear how to combine the best of the two worlds to get extremely small and efficient models. In this paper, we, for the first time, define the filter-level pruning problem for binary neural networks, which cannot be solved by simply migrating existing structural pruning methods for full-precision models. A novel learning-based approach is proposed to prune filters in our main/subsidiary network frame-work, where the main network is responsible for learning representative features to optimize the prediction performance, and the subsidiary component works as a filter selector on the main network. To avoid gradient mismatch when training the subsidiary component, we propose a layer-wise and bottom-up scheme. We also provide the theoretical and experimental comparison between our learning-based and greedy rule-based methods. Finally, we empirically demonstrate the effectiveness of our approach applied on several binary models, including binarizedNIN, VGG-11, and ResNet-18, on various image classification datasets. For bi-nary ResNet-18 on ImageNet, we use 78.6% filters but can achieve slightly better test error 49.87% (50.02%-0.15%) than the original model

Related Material


[pdf]
[bibtex]
@InProceedings{Xu_2019_CVPR,
author = {Xu, Yinghao and Dong, Xin and Li, Yudian and Su, Hao},
title = {A Main/Subsidiary Network Framework for Simplifying Binary Neural Networks},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}