iGOS++: integrated gradient optimized saliency by bilateral perturbations

Abstract

The black-box nature of the deep networks makes the explanation for "why" they make certain predictions extremely challenging. Saliency maps are one of the most widely-used local explanation tools to alleviate this problem. One of the primary approaches for generating saliency maps is by optimizing for a mask over the input dimensions so that the output of the network for a given class is influenced the most. However, prior work only studies such influence by removing evidence from the input. In this paper, we present iGOS++, a framework to generate saliency maps for blackbox networks by considering both removal and preservation of evidence. Additionally, we introduce the bilateral total variation term to the optimization that improves the continuity of the saliency map especially under high resolution and with thin object parts. We validate the capabilities of iGOS++ by extensive experiments and comparison against state-of-the-art saliency map methods. Our results show significant improvement in locating salient regions that are directly interpretable by humans. Besides, we showcased the capabilities of our method, iGOS++, in a real-world application of AI on medical data: the task of classifying COVID-19 cases from x-ray images. To our surprise, we discovered that sometimes the classifier is overfitted to the text characters printed on the x-ray images when performing classification rather than focusing on the evidence in the lungs. Fixing this overfitting issue by data cleansing significantly improved the precision and recall of the classifier.