Justice by Algorithm: The Limits of AI in Criminal Sentencing

AbstractCriminal justice systems have traditionally relied heavily on human decision-making, but new technologies are increasingly supplementing the human role in this sector. This paper considers what general limits need to be placed on the use of algorithms in sentencing decisions. It argues that, even once we can build algorithms that equal human decision-making capacities, strict constraints need to be placed on how they are designed and developed. The act of condemnation is a valuable element of criminal sentencing, and using algorithms in sentencing – even in an advisory role – threatens to undermine this value. The paper argues that a principle of “meaningful public control” should be met in all sentencing decisions if they are to retain their condemnatory status. This principle requires that agents who have standing to act on behalf of the wider political community retain moral responsibility for all sentencing decisions. While this principle does not rule out the use of algorithms, it does require limits on how they are constructed.Keywords: artificial intelligence (AI)criminal justiceFeinbergJoelpunishmentsentencing algorithms [I am very grateful to audiences at the Higher Seminar in Philosophy of Law at Uppsala University; the Political Theory Seminar at Stockholm University; and the workshop on “Ethics of AI in the Public Sector” at KTH Royal Institute of Technology for discussions on previous drafts of this paper; as well as to the anonymous reviewers from Criminal Justice Ethics for very helpful comments.][Disclosure Statement: No potential conflict of interest was reported by the author(s)].Notes1 Danziger, Levav and Avnaim-Pesso, “Extraneous Factors in Judicial Decisions.”2 Pamela McCroduck suggests that many members of disadvantaged groups may want to take their chances with an impartial computer over a (potentially biased) human judge. See McCorduck, Machines Who Think, 375.3 Yong, “A Popular Algorithm is No Better at Predicting Crimes Than Random People.”4 Angwin, Larson, Mattu, and Kirchner, “Machine Bias.” The question of whether algorithms can avoid objectionable forms of discrimination has been addressed in Davis and Douglas, “Learning to Discriminate: The Perfect Proxy Problem in Artificially Intelligent Criminal Sentencing.”5 Dressel and Farid, “The Accuracy, Fairness, and Limits of Predicting Recidivism.”6 One worry here is that there is no possible algorithm that can simultaneously meet various intuitively plausible criteria of fairness. See, for example, Chouldechova, “Fair Prediction with Disparate Impact.” I set this issue aside for the purposes of this paper, and assume that a fair algorithm is at least possible to construct. This might be because some of the purported criteria of fairness which cannot be met simultaneously are not, in fact, genuine moral requirements. Cf. Hedden, “On Statistical Criteria of Algorithmic Fairness;” Eva, “Algorithmic Fairness and Base Rate Tracking.”7 For the purposes of this paper, “sentencing decisions” will be taken to include not only the initial decision on severity and type of sentence given to criminals immediately after conviction, but also similar decisions made while punishment is being carried out (for example in parole hearings).8 A gap in research about how risk assessment scores are used in the criminal justice system more generally has been noted. See Law Society of England and Wales, “Algorithms in the Criminal Justice System.” 52.9 Skitka, Mosier and Burdick, “Does Automation Bias Decision-Making?”10 By “objective,” I mean that these factors do not consist of individuals' evaluations or psychological reactions. The contrast with subjective factors will be made in due course.11 The utilitarian Jeremy Bentham argues that the reduction of crime is the only legitimate end of criminal punishment. See Bentham, An Introduction to the Principles of Morals and Legislation, 170–203.12 Moore, Placing Blame.13 Ryberg, “Risk and Retribution.” Not all retributivists are resistant to risk-based sentencing. See Husak, “Why Legal Philosophers (Including Retributivists) Should be Less Resistant to Risk-Based Sentencing.”14 Cf. Chiao, “Predicting Proportionality,” 341–3.15 An algorithm that based its recommendations on previous judicial decisions may provide a useful proxy for these factors. An algorithm of this sort is imagined in ibid. Jesper Ryberg notes a dilemma for retributivists seeking to justify the use of machine-learning algorithms using existing cases as inputs. Either these algorithms will rely on a sample that is too small to give acceptable outcomes, or one that is too large to be easily constructed. See Ryberg, “Sentencing Disparity and Artificial Intelligence.”16 Abney, “Autonomous Robots and the Future of Just War Theory,” 347; Wallach and Vallor, “Moral Machines.”17 Duff, Punishment, Communication, and Community, 175–202; Lacey, “Socializing the Subject of Criminal Law;” Ristroph, “Desert, Democracy, and Sentencing Reform.”18 The retributive element might be thought to have this form. See Morris, The Future of Imprisonment.19 Up to the 1970s such wide discretion was given to judges in the US criminal justice system for instrumental reasons. See Berman, “Re-Balancing Fitness, Fairness, and Finality for Sentences,” 157–8.20 This might be based on the idea that desert is to some extent comparative, in the sense that what someone deserves (for example, the appropriate sentence on a retributivist account) might depend on what others have received. On this idea, See Miller, “Comparative and Noncomparative Desert.”21 For another notable example, see Wringe, An Expressive Theory of Punishment.22 Feinberg, “The Expressive Function of Punishment,” 400.23 Ibid., 397–8.24 Ibid., 404–8. We might add that condemnation might be welcomed by both instrumentalists (because the possibility of condemnation can serve as a useful disincentive) and retributivists (because the stigma that is produced by condemnation may form part of the harsh treatment that retributivists view as intrinsically valuable).25 Ibid., 406.26 Ibid.27 Honneth, The Struggle for Recognition.28 Ibid., 118–21.29 For the view that expressivists should support harsher sentencing for hate crimes, see Wellman, “A Defense of Stiffer Penalties for Hate Crimes,” 68.30 Nozick, Philosophical Explanations, 370.31 Cf. Shelby, Dark Ghettos, 240–241, where it is argued that it is the conviction (and not the sentencing) stage which involves an expressive element.32 Boonin, The Problem of Punishment, 176–9.33 For the view that censure is valuable, but not sufficient to justify punishment by itself, see Narayan, “Appropriate Responses and Preventive Benefits;” Hirsh, Censure and Sanctions, 6-19.34 It should be noted that certain theories of punishment that might be labelled “communicative” rather than expressive might also explain why there is a problem with certain uses of algorithms. These theories suggest that punishment should be a reciprocal act that requires some degree of rational engagement from those punished (see, for example, Duff, Punishment, Communication, and Community; Hampton, “The Moral Education Theory of Punishment”). Because certain algorithms may not be explicable to those who are sentenced, rational engagement might be impossible. While this idea warrants further attention, I cannot provide it within this paper.35 Fischer and Ravizza, Responsibility and Control, 12–4.36 For a helpful outline of these different forms of responsibility, see Jeppsson, “Accountability, Answerability and Attributibility.”37 Watson, “Two Faces of Responsibility,” 229.38 Sharkey, “Autonomous Weapons Systems, Killer Robots, and Human Dignity;” Sparrow, “Robots and Respect;” Sparrow, “Killer Robots,” 67; Taylor, “Who is Responsible for Killer Robots,” 232–3.39 This concept has emerged as the guiding principle for framing ongoing international negotiations on the regulation of LAWS. See United Nations Office at Geneva, “Report of the 2019 Session of the Group of Governmental Experts on Emerging Technologies in the Area of Lethal Autonomous Weapons Systems,” 25.40 See, for example, Sparrow, “Killer Robots.”41 Santoni de Sia and van den Hoven, “Meaningful Human Control over Autonomous Systems.”42 Taylor, “Who is Responsible for Killer Robots?,” 234.43 Explicability is a widely-recognized requirement of AI ethics, for various different sorts of reasons than the one outlined here. See Floridi and Cowls, “A Unified Framework of Five Principles for AI in Society,” 8. The importance of explicability more generally is discussed in Vredenburgh, “The Right to Explanation.” On the sort of explicability we want from algorithms, see Chiao, “Transparency at Sentencing;” Ryberg, “Sentencing and Algorithmic Transparency;” Ryberg and Petersen, “Sentencing and the Conflict between Algorithmic Accuracy and Transparency.”44 Roff and Moyes, “Meaningful Human Control, Artificial Intelligence and AutonomousWeapons.”45 Cf. Wellman, Rights Forfeiture and Punishment, 49.46 The complementary argument that punishment (rather than sentencing) should be undertaken by public agents for expressive reasons is given in Dorfman and Harel, “The Case Against Privatization,” 92–6.47 Cf. the distinction between direct and indirect delegation in Lawford-Smith, Not in Their Name, 117.48 Dorfman and Harel, “The Case Against Privatization,” 71–6.49 Ripstein, Force and Freedom, 190–8.50 Cordelli, The Privatized State, 159–71.51 This would be a “bottom-up” algorithm in the terms of Tasioulas, “AI and Robot Ethics,” 337.52 How strict these limits are will depend on the precise externalist principles we endorse. On Dorfman and Harel's account, for example, which requires that public agents defer to the public point of view, necessitates a community of practice – an institutional structure that allows the public point of view to be articulated and deferred to. See Dorfman and Harel, “The Case Against Privatization,” 82–3. It is unclear how this could genuinely be put in place when private companies are involved.53 Miller, 117.54 Ibid., 87.55 Ibid.56 This might be particularly true in the security sector. See Pattison, The Morality of Private War, 84–100.57 Shelby, Dark Ghettos, 238–48.