Automated Risk Classification and Outlier Detection

Abstract

Risk assessment is a common task present in a variety of problem domains, ranging from the assignment of premium classes to insurance applications, to the evaluation of disease treatments in medical diagnostics, situation assessments in battlefield management, state evaluations in planning activities, etc. Risk assessment involves scoring alternatives based on their likelihood to produce better or worse than expected returns in their application domain. Often, it is sufficient to evaluate the risk associated with an alternative by using a predefined granularity derived from an ordered set of risk-classes. Therefore, the process of risk assessment becomes one of classification. Traditionally, risk classifications are made by human experts using their domain knowledge to perform such assignments. These assignments will drive further decisions related to the alternatives. We address the automation of the risk classification process by exploiting risk structures present in sets of historical cases classified by human experts. We use such structures to pre-compile risk signatures that are compact and can be used to classify new alternatives. Specifically, we use dominance relationships, exploiting the partial ordering induced by the monotonic relationship between the individual features and the risk associated with a candidate alternative, to extract such signatures. Due to its underlying logical basis, this classifier produces highly accurate and defensible risk assignments. However, due to its strict applicability constraints, it covers only a small percentage of new cases. In response, we present a weaker version of the classifier, which incrementally improves its coverage without any substantial drop in accuracy. Although these approaches could be used as risk classifiers on their own, we found their primary strengths to be in validating the overall logical consistency of the risk assignments made by human experts and automated systems. We refer to potentially inconsistent risk assignments as outliers and present results obtained from implementing our technique in the problem of insurance underwriting