Reconceptualizing Rapid Responses as a Speededness Indicator in High-Stakes Assessments

ABSTRACT Previous research on rapid responding tends to implicitly consider examinees as either engaging in solution behavior or purely guessing. However, particularly in a high-stakes testing context, examinees perceiving that they are running out of time may consider the remaining items for less time than necessary to provide a fully informed response, but longer than a truly rapid guess. This partial consideration results in a response that misrepresents true ability, but with accuracy above the level of pure chance. To address this limitation of existing methodology, we propose an empirical approach that attempts to disentangle fully and partially informed responses to be used as a preliminary measure of the extent to which speededness may be distorting test score validity. We first illustrate and validate the approach using an experimental dataset in which the amount of time per item was manipulated. Next, applications of this approach are demonstrated using observational data in a more realistic context through four operational exams in which speededness is unknown.