Classification of Evaluation Measurement Taken Predicted Scores into Account

Abstract

Classifying performance evaluation is one of open problems in data mining and machine learning fields. We note that nearly all the existing evaluation measures ignore the predicted probabilities which are greatly significant in the process of classifiers’ evaluation. In this paper, we construct a weighted confusion matrix to reflect the information on predicted probabilities. In addition, based on the weighted confusion matrix, traditional evaluation measures, such as accuracy, precision, recall, F-measure, are redefined to taking predicted probabilities into account. Finally, properties of the re-written evaluation measures are investigated. Experimental results show that the re-defined evaluation measures are superior to traditional ones in terms of discrimination.