Evaluation of Class Activation Methods for Understanding Image Classification Tasks

Abstract

Machine Learning systems based on deep neural networks are powerful tools but their wide adoption has shown that both the designers and the users of DNN models must fight the barriers of understanding and controlling what has been learnt from data. To make such systems explainable and interpretable, model-specific post-hoc methods have been developed in the literature. This paper presents a family of such methods, Class Activation Mapping, used to explain the image classification process in Convolutional Neural Networks, and achieves a thorough evaluation of these methods. The analysis is done both a qualitative point of view, through the visualization of the activation heatmap of the image, and from a quantitative point of view, through several metrics that try to objectively quantify the relevant parts of an image which contributed to the classification. Several datasets are used to evaluate the discussed methods and a comparison between the obtained results is presented.