Explaining deep residual networks predictions with symplectic adjoint method

Abstract

Understanding deep residual networks (ResNets) decisions are receiving much attention as a way to ensure their security and reliability. Recent research, however, lacks theoretical analysis to guarantee the faithfulness of explanations and could produce an unreliable explanation. In order to explain ResNets predictions, we suggest a provably faithful explanation for ResNet using a surrogate explainable model, a neural ordinary differential equation network (Neural ODE). First, ResNets are proved to converge to a Neural ODE and the Neural ODE is regarded as a surrogate model to explain the decision-making attribution of the ResNets. And then the decision feature and the explanation map of inputs belonging to the target class for Neural ODE are generated via the symplectic adjoint method. Finally, we prove that the explanations of Neural ODE can be sufficiently approximate to ResNet. Experiments show that the proposed explanation method has higher faithfulness with lower computational cost than other explanation approaches and it is effective for troubleshooting and optimizing a model by the explanation.