Exploring the Granularity of Sparsity in Convolutional Neural Networks

Huizi Mao, Song Han, Jeff Pool, Wenshuo Li, Xingyu Liu, Yu Wang, William J. Dally; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2017, pp. 13-20

Granularity of sparsity affects the prediction accuracy of Deep Neural Network models. In this paper we quantitatively measure the accuracy-sparsity relationship with different grain sizes. The results validate the previous observations that larger grain size leads to worse accuracy. However, due to the index saving effect, coarse-grained sparsity is able to obtain similar or even better compression rates than fine-grained sparsity at the same accuracy threshold. Our analysis, which is based on the framework of recent sparse convolutional neural network(SCNN) accelerator, further demonstrates that it saves 30% - 35% of memory references compared with fine-grained ones.