On the making of crystal balls: Five lessons about simulation modeling and the organization of work

Abstract

Digital models that simulate the dynamics of a system are increasingly used to make predictions about the future. Although modeling has been central to decision-making under conditions of uncertainty across many industries for many years, the COVID-19 pandemic has made the role that models play in prediction and policymaking real for millions of people around the world. Despite the fact that modeling is a process through which experts use data and statistics to make sophisticated guesses, most consumers expect a model's predictions to be like crystal balls and provide perfect information about what the future will bring. Over the last decade, we have conducted a series of in-depth, longitudinal studies of digital modeling across several industries. From these studies, we share five lessons we have learned about modeling that demonstrate (1) why models are indeed not crystal balls and (2) why, despite their indeterminacy, people tend to treat them as crystal balls anyway. We discuss what each of these lessons can teach us about how to respond to the predictions made by COVID-19 models as well models of other stochastic processes and events about whose futures we wish to know today.