Analyzing Multiple-Choice-Multiple-Response Items Using Item Response Theory

Abstract

Multiple-choice-multiple-response (MCMR) items allow students to select as many responses as they think are correct to a given question stem. Using MCMR items can provide researchers and instructors with a richer and more complete picture of what students do and do not understand about a particular topic. Interpreting students’ MCMR responses is more nuanced than it is for single-response items. Unfortunately, many typical analyses of data from multiple-choice tests assume dichotomously-scored items, which eliminates the possibility of incorporating the rich information from students’ response patterns to MCMR items. We present a novel methodology for using a combination of item response theory models to analyze data from MCMR items. These methods could be applied to inform scoring models that incorporate partial credit for various response patterns.