Risk Assessments and Fairness Under Missingness and Confounding

Abstract

Fairness in machine learning has become a significant area of research as risk assessments and other algorithmic decision-making systems are increasingly used in high-stakes applications such as criminal justice, consumer lending, and child welfare screening decisions. Two significant challenges to achieving fair decision-making systems are 1) access to the protected attribute may be limited and 2) the outcome may be confounded or selectively observed depending on the historical data generating process. To address the former challenge, we propose two methods for overcoming limited access to the protected attribute and empirically evaluate their success on three datasets. To address the later challenge, we develop counterfactual risk assessments that account for the effect of historical interventions on the outcome. We analyze the performance of our counterfactual risk assessments in criminal sentencing decisions in Pennsylvania. We compare our model against observational risk assessments.