Randomized Channel-pass Mask for Channel-wise Explanation of Black-box Models

Hirotaka Hachiya, Daiki Nisawa; Proceedings of the Asian Conference on Computer Vision (ACCV), 2024, pp. 3756-3770

In recent years, there has been active research on interpreting the classification results of deep models. Among these methods, MC-RISE enables pixel-color-wise interpretation based on the models output for images where pixels have been randomly replaced with a predetermined color. However, this approach requires manually preparing appropriate color candidates. This study proposes a pixel-channel-wise interpretation method, using a Randomized Channel-pass Mask (RaCM), which directly evaluates the importance of the original RGB values of an image through randomly generated masks that pass or exclude color channels of each pixel. Experiments are conducted using the German Traffic Sign Recognition Benchmark and ImageNet datasets. The effectiveness of the proposed method is demonstrated through evaluation metrics such as Insertion, Deletion, and Average DCC.