Educational Measurement-Issues and Practice最新文献

Random Equating (RE) and Heuristic Approach (HA) are two linking procedures that may be used to compare the scores of individuals in two tests that measure the same latent trait, in conditions where there are no common items or individuals. In this study, RE—that may only be used when the individuals taking the two tests come from the same population—was used as a benchmark for evaluating HA, which, in contrast, does not require any distributional assumptions. The comparison was based on both simulated and empirical data. Simulations showed that HA was good at reproducing the link shift connecting the difficulty parameters of the two sets of items, performing similarly to RE under the condition of slight violation of the distributional assumption. Empirical results showed satisfactory correspondence between the estimates of item and person parameters obtained via the two procedures.

Item position effect (IPE) refers to situations where an item performs differently when it is administered in different positions on a test. The majority of previous research studies have focused on investigating IPE under linear testing. There is a lack of IPE research under adaptive testing. In addition, the existence of IPE might violate Item Response Theory (IRT)’s item parameter invariance assumption, which facilitates applications of IRT in various psychometric tasks such as computerized adaptive testing (CAT). Ignoring IPE might lead to issues such as inaccurate ability estimation in CAT. This article extends research on IPE by proposing and evaluating approaches to controlling position effects under an item-level computerized adaptive test via a simulation study. The results show that adjusting IPE via a pretesting design (approach 3) or a pool design (approach 4) results in better ability estimation accuracy compared to no adjustment (baseline approach) and item-level adjustment (approach 2). Practical implications of each approach as well as future research directions are discussed as well.

Growth-to-standard models evaluate student growth against the growth needed to reach a future standard or target of interest, such as proficiency. A common growth-to-standard model involves comparing the popular Student Growth Percentile (SGP) to Adequate Growth Percentiles (AGPs). AGPs follow from an involved process based on fitting a series of nonlinear quantile regression models to longitudinal student test score data. This paper demystifies AGPs by deriving them in the more familiar linear regression framework. It further shows that unlike SGPs, AGPs and on-track classifications based on AGPs are strongly related to status. Lastly, AGPs are evaluated in terms of their classification accuracy. An empirical study and analytic derivations reveal AGPs can be problematic indicators of students’ future performance with previously not proficient students being more likely incorrectly flagged as not on-track and previously proficient students as on track. These classification errors have equity implications at the individual and school levels.

Double-scoring constructed-response items is a common but costly practice in mixed-format assessments. This study explored the impacts of Targeted Double-Scoring (TDS) and random double-scoring procedures on the quality of psychometric outcomes, including student achievement estimates, person fit, and student classifications under various conditions that reflect operational performance assessments. Using a simulation study, our results suggest no notable advantages for TDS over the random double-scoring approach across various psychometric outcomes, regardless of conditions related to student misfit, rater misfit, and rater severity. This study holds significant implications for mixed-format assessments, offering insights into a comprehensive evaluation of double-scoring methods. We recommend that researchers consider these findings when considering among double-scoring procedures.

The work of educational measurement is a highly collaborative endeavor that brings together professionals from many disciplines. While the introduction of the “Foundational Competencies in Educational Measurement” acknowledges this, the explanation of the framework itself falls short in acknowledging the competencies and skills of those from disciplines other than psychometrics, such as content development professionals (CDPs). Therefore, it is unable to sufficiently address the nature of validation work or other work not led by psychometricians. It also underexplores the vital competencies that underlie effective collaboration. As a result, it defines the competencies of psychometric work instead of the larger field of educational measurement.

The Task Force on Foundational Competencies in Educational Measurement has produced a set of foundational competencies and invited comment on the document. The students and faculty at the University of Massachusetts Amherst provide their comments and critique of the proposed competencies. Both students and faculty agree that there needs to be more specificity regarding the purpose of the document, the nature of the data used to produce the document, and the definition of the relevant terms. Additionally, attention should be paid to the international context, and the role of artificial intelligence and machine learning. The authors acknowledge the contribution of the draft of the foundational competencies and look forward to more conversation regarding this topic.