Investigating Calibration and Corruption Robustness of Post-hoc Pruned Perception CNNs: An Image Classification Benchmark Study

Pallavi Mitra, Gesina Schwalbe, Nadja Klein; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 3542-3552

Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance in many computer vision tasks. However high computational and storage demands hinder their deployment into resource-constrained environments such as embedded devices. Model pruning helps to meet these restrictions by reducing the model size while maintaining superior performance. Meanwhile safety-critical applications pose more than just resource and performance constraints. In particular predictions must not be overly confident i.e. provide properly calibrated uncertainty estimations (uncertainty calibration) and CNNs must be robust against corruptions like naturally occurring input perturbations (natural corruption robustness). This work investigates the important trade-off between uncertainty calibration natural corruption robustness and performance for current state-of-research post-hoc CNN pruning techniques in the context of image classification tasks. Our study reveals that post-hoc pruning substantially improves the model's uncertainty calibration performance and natural corruption robustness sparking hope for safe and robust embedded CNNs. Furthermore uncertainty calibration and natural corruption robustness are not mutually exclusive targets under pruning as evidenced by the improved safety aspects obtained by post-hoc unstructured pruning with increasing compression.