User-centric explainable AI: design and evaluation of an approach to generate coherent counterfactual explanations for structured data

Abstract

ABSTRACT Many Artificial Intelligence (AI) systems are black boxes, which hinders their deployment. Explainable AI (XAI) approaches which automatically generate counterfactual explanations aim to assist users in scrutinising AI decisions. One property of explanations crucial for their acceptance by users is their coherence. Users perceive counterfactual explanations as coherent if they present a realistic/typical counterfactual scenario that is suitable to explain the factual situation. We design an optimisation-based approach to generate coherent counterfactual explanations applicable to structured data. We demonstrate its applicability and rigorously evaluate its efficacy through functionally grounded and human-grounded evaluation. Results suggest that our approach indeed produces counterfactual explanations that are perceived as coherent by users. More specifically, they are perceived as more realistic, typical, and feasible than state-of-the-art explanations.