Taking Goodhart’s Law Meta: Gaming, Meta-Gaming, and Hacking Academic Performance Metrics

Abstract

demic performance metrics and misconduct. One is that Goodhart’s law (1975) concerns more than simply the idea of individual responsiveness to pressures from societal policies, for example, central bank monetary policies employ economic performance measures as standards of regulation and control in banking. The other concerns how we might exploit what more there is to Goodhart’s law to probe the character of “mis”conduct, as individuals and organizations adapt to, and comply with, academic performance metrics institutionalized as standards. Contrast this with “bad” conduct, as individuals and organizations cynically attempt to “game” or “exploit” the system to achieve a better evaluation than their performance warrants. Along with other chapters in this volume (Csiszar, this volume, chapter 1; Power, this volume, chapter 3), I suggest Goodhart’s law describes conditions that not only undermine the representational success in modeling causal order in human social systems, but also the operation of the law inverts the causal order. Conversions of metrics into standards not only invite “gaming the system,” they also practically construct “gaming” as the new form of practice, rendering the original product or practice to be measured as a “side effect” in the new causal order. Or, as Wouters (this volume, chapter 4) urges, we must distinguish gaming the system from properly functioning in an inverted system. It is thus problematic to moralize and shame socalled “predatory” practices as if it were clear what constitutes ethical, nonpredatory practice in social worlds where Goodhart’s law operates. Goodhart’s lesson was that such measures are selfdefeating because they invite “mis”conduct. If people respond to standards as intended, the measure ceases to represent and record the primary target performance and comes to measure only compliance or conformity to the standard. The critique cuts deeper. As Lucas’s (1976) critique of macroeconometric 5 Taking Goodhart’s Law Meta: Gaming, MetaGaming, and Hacking Academic Performance Metrics