Is Our Continual Learner Reliable? Investigating Its Decision Attribution Stability through SHAP Value Consistency

Yusong Cai, Shimou Ling, Liang Zhang, Lili Pan, Hongliang Li; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 5568-5575

In this work we identify continual learning (CL) methods' inherent differences in sequential decision attribution. In the sequential learning process inconsistent decision attribution may undermine the interpretability of a continual learner. However existing CL evaluation metrics as well as current interpretability methods cannot measure the decision attribution stability of a continual learner. To bridge the gap we introduce Shapley value a well-known decision attribution theory and define SHAP value consistency (SHAPC) to measure the consistency of a continual learner's decision attribution. Furthermore we define the mean and the variance of SHAPC values namely SHAPC-Mean and SHAPC-Var to jointly evaluate the decision attribution stability of continual learners over sequential tasks. On Split CIFAR-10 Split CIFAR-100 and Split TinyImageNet we compare the decision attribution stability of different CL methods using the proposed metrics providing a new perspective for evaluating their reliability.