Creating effective decision aids for complex tasks

Engineering design tasks require designers to continually compare, weigh, and choose among many complex alternatives. The quality of these selection decisions directly impacts the quality, cost, and safety of the final product. Because of the high degree of uncertainty in predicting the performance of alternatives while they are still just sketches on the drawing board, and the high cost of poor choices, mathematical decision methods incorporating uncertainty have long held much appeal for product designers, at least from a theoretical standpoint. Yet, such methods have not been widely adopted in practical settings. The goals of this work are to begin understanding why this is so and to identify future questions that may lead to solutions. This paper summarizes the results of several studies by the authors: two laboratory studies in which we asked product designers to use various mathematical models to compare and select design alternatives, and a set of ethnographic studies in which we observed product designers as they worked so that we could better understand their actual practices and needs during decision making. Based on these studies, we concluded that the mathematical models, as formulated, are not well suited to designers' needs and approaches. We propose a research agenda for developing new approaches that combine decision theoretic and user-centered methods to create tools that can make product designers' decision making work easier, more systematic, more effective, and more reportable.