Test Takers’ Response Tendencies in Alternative Item Formats: A Cognitive Science Approach

Abstract

ABSTRACT We investigated how item formats influence test takers’ response tendencies under uncertainty. Adult participants solved content-equivalent math items in three formats: multiple-selection multiple-choice, grid with forced-choice (true-false) options, and grid with non-forced-choice options. Participants showed a greater tendency to commit (rather than omit) responses in the grid items, in both forced-choice and non-forced-choice types, compared to the multiple-selection multiple-choice items. These findings relate to the theoretical framework of affordances, which predicts that the design of interactive artifacts can shape one’s perception of action opportunities. The results of a signal detection analysis provided additional evidence that the item formats affected participants’ response bias. The current research suggests that cognitive science principles could provide an in-depth understanding of test takers’ cognition in new item formats.