Solving the enigma: Enhancing faithfulness and comprehensibility in explanations of deep networks

Abstract

The accelerated progress of artificial intelligence (AI) has popularized deep learning models across various domains, yet their inherent opacity poses challenges, particularly in critical fields like healthcare, medicine, and the geosciences. Explainable AI (XAI) has emerged to shed light on these ’black box’ models, aiding in deciphering their decision-making processes. However, different XAI methods often produce significantly different explanations, leading to high inter-method variability that increases uncertainty and undermines trust in deep networks’ predictions. In this study, we address this challenge by introducing a novel framework designed to enhance the explainability of deep networks through a dual focus on maximizing both accuracy and comprehensibility in the explanations. Our framework integrates outputs from multiple established XAI methods and leverages a non-linear neural network model, termed the ‘Explanation optimizer,’ to construct a unified, optimal explanation. The optimizer uses two primary metrics — faithfulness and complexity — to evaluate the quality of the explanations. Faithfulness measures the accuracy with which the explanation reflects the network’s decision-making, while complexity assesses the comprehensibility of the explanation. By balancing these metrics, the optimizer provides explanations that are both accurate and accessible, addressing a central limitation in current XAI methods. Through experiments on multi-class and binary classification tasks in both 2D object and 3D neuroscience imaging, we validate the efficacy of our approach. Our explanation optimizer achieved superior faithfulness scores, averaging 155% and 63% higher than the best-performing individual XAI methods in the 3D and 2D applications, respectively, while also reducing complexity to enhance comprehensibility. These results demonstrate that optimal explanations based on specific quality criteria are achievable, offering a solution to the issue of inter-method variability in the current XAI literature and supporting more trustworthy deep network predictions.