Classical Item Analysis from a Signal Detection Perspective

Abstract

A conceptualization of multiple-choice exams in terms of signal detection theory (SDT) leads to simple measures of item difficulty and item discrimination that are closely related to, but also distinct from, those used in classical item analysis (CIA). The theory defines a “true split,” depending on whether or not examinees know an item, and so it provides a basis for using total scores to split item tables, as done in CIA, while also clarifying benefits and limitations of the approach. The SDT item difficulty and discrimination measures differ from those used in CIA in that they explicitly consider the role of distractors and avoid limitations due to range restrictions. A new screening measure is also introduced. The measures are theoretically well-grounded and are simple to compute by hand calculations or with standard software for choice models; simulations show that they offer advantages over traditional measures.