InfoMoD: Information-theoretic Model Diagnostics

Abstract

Validating and debugging machine learning models is done by testing them on unseen data. Analyzing model performance on various subsets of the data is critical for fairness, trust, bias detection and explainablility. In this paper, we describe a new way to do this. Our solution, called InfoMoD, applies recent work in information-theoretic data summarization to the problem of model diagnostics. Using real-life datasets, we show how InfoMod concisely describes how a model performs across different subsets of the data and produces expected performance indicators for individual test instances.