Plausible Cause: Explanatory Standards in the Age of Powerful Machines

INTRODUCTIONSuppose, in the near future, that police start using an algorithmic tool-the Contraband Detector-to locate residences likely to contain illegal weapons. When the tool was first developed, its outputs were thirty percent accurate. With time, however, machine learning refined the tool.1 Now its accuracy rate hovers around eighty percent, and data scientists, having recently “audited” the Contraband Detector,2 report that the tool’s performance will only continue to improve. When the tool locates a suspicious residence, it does not explain why; it simply displays an address. And because of the tool’s complexity-it draws on more than one hundred input-variables- officers have no idea which variables are determinative in a given case.3Here is the puzzle. Imagine the Contraband Detector, deployed in New York City, turns up “285 Court St., Apt. 2L,” prompting the NYPD to seek a search warrant. When the judge asks about probable cause, the officers point to one, and only one, fact: the tool’s performance rate.4 Should the judge sign the warrant? Or better yet: Could the judge’s role in the process simply be eliminated-at least in principle-such that any time the tool identifies a suspicious residence, a search warrant issues automatically?5 In other words, suppose the next generation of tool, operating on the same logic, is not a Contraband Detector, but an Automatic Warrant Machine. Assuming the tool continues to perform at a high level of statistical precision, would its use-in lieu of judicial oversight-be consistent with the Fourth Amendment?There is a powerful and widespread intuition that the answer to these questions is no.6 Performance aside, blind reliance on an algorithmic tool feels uncomfortable. It misses the point of particularized suspicion.7 But why? On its face, probable cause would seem to depend on the probability that a “person[ ], house[ ], paper[ ] or effect[ ]” is linked to wrongdoing.8 In the example, it is eighty percent probable that 285 Court St., Apt. 2L contains an illegal weapon. So probable cause, literally construed, should be satisfied.I propose a simple solution to this puzzle. For probable cause to be satisfied, an inference of wrongdoing must be plausible-the police must be able to explain why observed facts give rise to the inference.9 And judges must have an opportunity to scrutinize that explanation: to test its overall intelligibility; to weigh it against the best innocent account on the other side; and to evaluate its consistency with background values, flowing from the Constitution, from general legality principles, and from other sources of positive law.10This hardly means that prediction tools have no place in policing or in other areas of governance. It means, rather, that their role is to aid human reasoning, not to supplant it.11 Outputs from prediction tools, like outputs from other detection instruments, such as drug dogs,12 can certainly be among the facts that police adduce-in an explanatory fashion-to anchor claims of wrongdoing. For that process to work, however, a tool’s outputs must be intelligible. Black-box tools will not do. Nor will transparent tools with outputs too complex for a human to trace.13Although the Contraband Detector, as imagined, exceeds current technology, the trend it reflects-the blossoming of data-driven prediction tools in the criminal justice system-is hardly science fiction. In many jurisdictions, judges have already begun to rely heavily on prediction tools that predict the likelihood of flight or recidivism for bail and sentencing purposes,14 a practice recently upheld by the Wisconsin Supreme Court.15 Likewise, the first wave of suspicion tools have recently been adopted by police departments, often to help officers assess individuals’ “threat scores” while on patrol.16 At present, the technology is crude; no hyper-precise detection tool, able to predict the presence of contraband eighty percent of the time, yet exists. …