Lexical Similarity in Grid Questions: An Experiment of the Effect of “Same Leading” on Respondent Behaviors

Abstract

Previous research on grid questions has mostly focused on their comparability with the item-by-item method and the use of shading to help respondents navigate through a grid. This study extends prior work by examining whether lexical similarity among grid items affects how respondents answer the questions in an experiment where we manipulated lexical similarity by adding the same leading phrase to the items in a grid (“same leading” condition). Both interpretative heuristics and survey satisficing theory suggest that such a design could increase non-differentiated answers. Based on an experiment using respondents from a Chinese opt-in panel ( n = 2,000+), we confirmed more non-differentiation in the same leading condition (than the control) for low-education respondents, while the effects were not significant for the others. This study adds to the literature that incidental survey design features can make an impact and highlights the need for more research into the effects of wording styles.