Remedies against bias in analytics systems

Abstract

ABSTRACT Advances in IT offer the possibility to develop ever more complex predictive and prescriptive systems based on analytics. Organizations are beginning to rely on the outputs from these systems without inspecting them, especially if they are embedded in the organization’s operational systems. This reliance could be misplaced unethical or even illegal if the systems contain bias. Data, algorithms and machine learning methods are all potentially subject to bias. In this article we explain the ways in which bias might arise in analytics systems, present some examples, and give some suggestions as to how to prevent or reduce it. We use a framework inspired by the work of Hammond, Keeney and Raiffa on psychological traps in human decision-making. Each of these traps “translates” into a potential type of bias for an analytics-based system. Fortunately, this means that remedies to reduce bias in human decision-making also translate into potential remedies for algorithmic systems.