An AI Framework for Modelling and Evaluating Attribution Methods in Enhanced Machine Learning Interpretability

Abstract

In this paper, we propose a general methodology for estimating the degree of the attribution methods precision and generality in machine learning interpretability. Additionally, we propose a technique to measure the attribution consistency between two attribution methods. In our experiments, we focus on the two well-known model agnostic attribution methods, SHAP and LIME, then we evaluate them on two real applications in the attack detection field. Our proposed methodology highlights the fact that both LIME and SHAP are lacking precision, generality, and consistency. Therefore, more inspection is needed in the attribution research field.