Assessing Fair Machine Learning Strategies Through a Fairness-Utility Trade-off Metric

Abstract

Due to the increasing use of artificial intelligence for decision making and the observation of biased decisions in many applications, researchers are investigating solutions that attempt to build fairer models that do not reproduce discrimination. Some of the explored strategies are based on adversarial learning to achieve fairness in machine learning by encoding fairness constraints through an adversarial model. Moreover, it is usual for each proposal to assess its model with a specific metric, making comparing current approaches a complex task. In that sense, we defined a utility and fairness trade-off metric. We assessed 15 fair model implementations and a baseline model using this metric, providing a systemically comparative ruler for other approaches.