Assessing Differential Statement Functioning in Polytomous Multidimensional Pairwise Comparison Items.

Abstract

Multidimensional pairwise comparison (MPC) items have been widely used to assess career interest, value and personality to avoid response bias in educational sectors. In reality, a statement in an MPC item may have different utilities for different groups, which is referred to as differential statement functioning (DSF). Few studies have been investigated DSF assessment. Based on a Rasch model for MPC items, this study adapts three methods to detect DSF for polytomous MPC items: the equal-mean-utility (EMU) method, the all-other-statement (AOS) method and the constant-statement (CS) method. Simulation study was conducted to evaluate the recovery of parameters as well as the performance of the proposed methods. Results showed that when the test contains DSF statement(s), the CS method where one or more DSF-free statements are chosen as an anchor will yield accurate estimates and perform well for DSF assessment. An empirical example of career interest assessment was provided. .