Different methods elicit different belief distributions.

Abstract

When eliciting people's forecasts or beliefs, you can ask for a point estimate-for example, what is the most likely state of the world?-or you can ask for an entire distribution of beliefs-for example, how likely is every possible state of the world? Eliciting belief distributions potentially yields more information, and researchers have increasingly tried to do so. In this article, we show that different elicitation methods elicit different belief distributions. We compare two popular methods used to elicit belief distributions: Distribution Builder and Sliders. In 10 preregistered studies (N = 14,553), we find that Distribution Builder elicits more accurate belief distributions than Sliders, except when true distributions are right-skewed, for which the results are mixed. This result holds when we assess accuracy (a) relative to a normative benchmark and (b) relative to participants' own beliefs. Our evidence suggests that participants approach these two methods differently: Sliders users are more likely to start with the lowest bins in the interface, which in turn leads them to put excessive mass in those bins. Our research sheds light on the process by which people construct belief distributions while offering a practical recommendation for future research: All else equal, Distribution Builder yields more accurate belief distributions. (PsycInfo Database Record (c) 2024 APA, all rights reserved).