Behind the curtain: lessons learned from a Wizard of Oz field experiment

Abstract

Large scale, organization-wide groupware systems are high risk development efforts. Requirements gathering and early evaluation are constrained by the need for the system to attain a critical mass of users and content. One approach to mitigate this risk is to employ Wizard of Oz style system simulations during the requirements gathering phase. While this method has historically been used to test quasi-functional system prototypes, we have found it to be a useful method for assessing organizational feasibility. After introducing this adaptation of the Wizard of Oz method, our approach is grounded with a case study in which it is was successfully used as a proof-of-concept requirements gathering exercise for an expertise recommender and organizational memory system project. Transferable lessons learned are distilled and discussed.