No pain, no gain: The structure and consequences of question difficulty in a management accounting course

Abstract

An unrecognized choice made by accounting instructors is the degree of difficulty presented to students. Even if assessment is limited to multiple-choice materials, instructors can pick questions that vary from easy to very difficult. To do this, instructors depend on difficulty classifications provided by textbook authors and publishers. Using computer captured results from student efforts, this paper tests the integrity of these difficulty categories. This effort considers the impact of different grade consequences, different scoring systems, student aptitude and prior question exposure. The extent difficulty levels translate into student success variation is reported. The results generally support the publisher’s classifications. Nonetheless, interesting variations in the degree of these differences can be attributed to the context and consequences of question answering, as well as to the type of student asked to answer.