An Empirical Study of Data-Free Quantization's Tuning Robustness

Hong Chen, Yuxuan Wen, Yifu Ding, Zhen Yang, Yufei Guo, Haotong Qin; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022, pp. 172-178

Abstract


Deep convolutional neural networks are now performing increasingly superior in various fields, while the network parameters are getting massive as the advanced neural networks tend to be deeper. Among various model compression methods, quantization is one of the most potent approaches to compress neural networks by compacting model weights and activations to lower bit-width. The data-free quantization method is also proposed, which is specialized for some privacy and security scenarios and enables quantization without access to real data. In this work, we find that the tuning robustness of existing data-free quantization is flawed, progressing an empirical study and determining some hyperparameter settings that can converge the model stably in the data-free quantization process. Our study aims to evaluate the overall tuning robustness of the current data-free quantization system, which is existing methods are significantly affected by parameter fluctuations in tuning. We also expect data-free quantification methods with tuning robustness to appear in the future.

Related Material


[pdf]
[bibtex]
@InProceedings{Chen_2022_CVPR, author = {Chen, Hong and Wen, Yuxuan and Ding, Yifu and Yang, Zhen and Guo, Yufei and Qin, Haotong}, title = {An Empirical Study of Data-Free Quantization's Tuning Robustness}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops}, month = {June}, year = {2022}, pages = {172-178} }