NoSimple: Data Bias Evaluation Metrics

Abstract

Simple objects are defined as objects invariably correctly classified by all outlier detectors. Its presence impairs performance of binary classifiers such as ROC or F1 score. A large number of simple objects falsely improve performance of binary classifiers when evaluated by ROC or F1 score. This impairs reliability of classifier evaluation. This manuscript proposes evaluation without simple objects (NoSimple). NoSimple preprocesses data to factor in simple objects by removing the simple objects for the evaluation phase. Experiments with 30 realworld datasets demonstrate that NoSimple significantly reduced the average ROC of all classifiers by $0.04 \sim 0.06$. NoSimple is most effective when the percentage of simple objects exceeds $30{\% }$. By introducing a new method to reliably evaluate outlier classifiers, NoSimple has the potential to revolutionize evaluation metrics and has a multitude of applications in data science research.