Allowing Humans to Interactively Guide Machines Where to Look Does Not Always Improve Human-AI Team's Classification Accuracy

Giang Nguyen, Mohammad Reza Taesiri, Sunnie S. Y. Kim, Anh Nguyen; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 8169-8175

Via thousands of papers in Explainable AI (XAI) attention maps and feature attribution maps have been established as a common means for finding how important each input feature is to an AI's decisions. It is an interesting unexplored question whether allowing users to edit the feature importance at test time would improve a human-AI team's accuracy on downstream tasks. In this paper we address this question by leveraging CHM-Corr a state-of-the-art ante-hoc explainable classifier that first predicts patch-wise correspondences between the input and training-set images and then base on them to make classification decisions. We build CHM-Corr++ an interactive interface for CHM-Corr enabling users to edit the feature attribution map provided by CHM-Corr and observe updated model decisions. Via CHM-Corr++ users can gain insights into if when and how the model changes its outputs improving their understanding beyond static explanations. However our user study with 18 users who performed 1400 decisions finds no statistical significance that our interactive approach improves user accuracy on CUB-200 bird image classification over static explanations. This challenges the hypothesis that interactivity can boost human-AI team accuracy and raises needs for future research. We open-source CHM-Corr++ an interactive tool for editing image classifier attention. We release code and data on github.