Experimenting with List Experiments: Interviewer Effects and Immigration Attitudes

Abstract

List experimentation is a common survey methodology that purports to reduce or eliminate social desirability bias. While some studies have assessed list experimentation’s effectiveness in achieving that goal, to our knowledge, this is the first ever experimental evaluation of interviewer effects on list experiment performance. We embedded a list experiment about immigration attitudes in an in-person survey administered to 718 white respondents. Randomly assigning Caucasian and Latinx interviewers, we find strong evidence that responses to the list experiment differed by interviewer ethnicity, thus failing to fully eliminate social desirability bias. A follow-up survey of 1,460 online respondents revealed similar difference-in-differences when merely priming the ethnic identities of survey researchers through pictures. The results of this study shed light on patterns of interpersonal communication about sensitive issues and how social context shapes the reporting of political attitudes, even when methodology specifically meant to mute sensitivity biases is employed.