Seg-XRes-CAM: Explaining Spatially Local Regions in Image Segmentation

Syed Nouman Hasany, Caroline Petitjean, Fabrice Mériaudeau; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 3733-3738

Abstract


While many post-hoc model interpretability techniques exist for image classification, image segmentation has not received the same attention. An extension of Grad-CAM, Seg-Grad-CAM was proposed as a local interpretability technique for image segmentation. In this paper, we highlight that by virtue of its design, Seg-Grad-CAM does not utilize spatial information when it comes to generating explanations for regions within a segmentation map. Taking inspiration from HiResCAM, we propose Seg-XRes-CAM in order to solve this problem. We verify the utility of our proposed method by visually comparing explanations generated from Seg-Grad-CAM and Seg-XRes-CAM against a model-agnostic, perturbation-based method, RISE. The code is available at https://github.com/Nouman97/Seg_XRes_CAM.

Related Material


[pdf]
[bibtex]
@InProceedings{Hasany_2023_CVPR, author = {Hasany, Syed Nouman and Petitjean, Caroline and M\'eriaudeau, Fabrice}, title = {Seg-XRes-CAM: Explaining Spatially Local Regions in Image Segmentation}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops}, month = {June}, year = {2023}, pages = {3733-3738} }