Quality indicators for Delphi studies

Abstract

The Delphi method is a technique of social research that seeks to obtain a reliable group opinion from experts. It was first created for military purposes in the mid-1950s. Since then, its use in the scientific field has continued to spread to different disciplines and aims. Despite this expansion, however, not set of indicators of the quality of Delphi studies has yet to be developed that might provide the reader—whether an expert in the technique or not—with some framework of reference whereby to gauge what credibility should be afforded to the results of the study. In this paper, following a thorough review of the literature on the criteria used to assess Delphi studies and the items of evaluation recommended for inclusion in Delphi reports, we determine what characteristics a quality evaluation indicator for this technique should have and propose a battery of indicators based on these characteristics, which should for preference be included in the final report of a Delphi study. The proposed indicators focus on three areas that are particularly relevant to the quality of Delphi research: the quality of the panel of participating experts, the way in which relevant information is obtained from the experts, and the quality of the interaction generated among the experts.