Interpretation of Deep CNN Recognition with Filter Space Clustering in Feature Extraction and Reconstruction

Sukhan Lee, Naeem Ul Islam; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2019, pp. 32-35

Interpreting a deep Convolutional Neural Network (CNN) involves identifying the features in a hierarchy of layers that contribute to recognition. Although the current approaches serve as methods to interpret a deep CNN, further advancement is required for a more accurate and efficient way of understanding how a hierarchy of features formed by a deep CNN contributes to recognition. In this paper, we propose attaching a feedback CNN to a pretrained feedforward CNN as a means of learning how recognition is performed by the feedforward CNN. In other words, the features reconstructed in a hierarchy of the feedback CNN represent those learned by the feedforward CNN. By analyzing how clusters are formed in the layers of feature spaces in the feedback CNN, we can interpret which features critically contribute to recognition. It also helps to evaluate whether or not recognition is done successfully. In order to show this, we experimentally verify the capabilities of the proposed approach in terms of identifying incorrectly recognized input data by pinpointing the source of the error in feature spaces. Experiments conducted on the ModelNet datasets indicate that the proposed approach offers an extended capability of interpreting a deep CNN as described above with higher accuracy than conventional approaches.