Assessing Visually-Continuous Corruption Robustness of Neural Networks Relative to Human Performance

Huakun Shen, Boyue Hu, Krzysztof Czarnecki, Lina Marsso, Marsha Chechik; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 6300-6310

Neural Networks (NNs) have surpassed human accuracy in image classification on ImageNet yet they often lack robustness against image corruption i.e. corruption robustness with such robustness being seemingly effortless for human perception. In this paper we propose visually-continuous corruption robustness (VCR) - an extension of corruption robustness to allow assessing it over the wide and continuous range of changes that correspond to the human perceptive quality (i.e. from the original image to the full distortion of all perceived visual information) along with two novel human-aware metrics for NN evaluation. To compare VCR of NNs with human perception we conducted extensive experiments on 14 commonly used image corruptions with 7718 human participants and state-of-the-art robust NN models with different training objectives (e.g. standard adversarial corruption robustness) different architectures (e.g. convolution NNs vision transformers) and different amounts of training data augmentation. Our study showed that: 1) assessing robustness against continuous corruption can reveal insufficient robustness undetected by existing benchmarks; as a result 2) the gap between NN and human robustness is larger than previously known; and finally 3) some image corruptions have a similar impact on human perception offering opportunities for more cost-effective robustness assessments.