What Influences Trust in Survey Results? Evidence from a Vignette Experiment

Abstract

Today, there are more survey results available than ever before. This increase in survey data is, however, accompanied by a decline in survey quality. Thus, it is more likely than in the past that citizens and politicians get a biased picture of public opinion when relying on survey results. Those misperceptions can have worrying consequences for political discourse and decision-making. With the present study, we aim to investigate to what extent the public draws on survey quality information when evaluating the trustworthiness of a survey result. To explore this research question, we implemented a vignette experiment in an online panel survey (n = 3,313) in which each respondent was confronted with four different, randomly assigned descriptions of a survey and then asked to evaluate the trustworthiness of the respective survey result. The survey descriptions varied regarding the methodological information provided (i.e., sample size, sampling method, and sample balance). The results showed that survey quality information only had a minor effect on the perceptions of trust compared to respondents’ characteristics, such as pre-existing opinions on the topic or general trust in science. Yet, trust in the survey result was significantly influenced by the sample size and sample balance, but not by the sampling method. Finally, in line with information processing theory, the relevance of survey quality information increases with the cognitive abilities of the respondent.