Educational Measurement-Issues and Practice最新文献

The cover of this issue featured “The Increasing Impact of EM:IP” by Zhongmin Cui, the journal's editor. Cui elaborated on the significance of the impact factor for Educational Measurement: Issues and Practice (EM:IP), one of the most widely recognized metrics for evaluating a journal's influence and prestige. The impact factor, which measures how frequently a journal's articles are cited over a specific period, serves as a critical tool for researchers, institutions, and funding bodies in assessing the relevance and significance of published work.

Cui noted the challenges in measuring a journal's influence, stating, “As measurement professionals, we are well aware of the difficulties in quantifying almost anything, including the impact of a journal. However, even imperfect metrics, if carefully designed, can provide valuable insights for users making informed decisions.”

He cited EM:IP’s latest journal impact factor of 2.7 (Wiley, 2024), which was calculated based on citations from the previous two years. Acknowledging that this figure might not seem substantial, Cui emphasized that it represents a significant milestone in the journal's history. “The visualization we created illustrates a steady, consistent upward trend in EM:IP’s impact factor over the past decade. This growth reflects our ongoing commitment to publishing high-quality, impactful research that resonates with both scholars and practitioners,” he added.

Cui also stressed the growing influence of EM:IP in the field of educational and psychological measurement. He credited this achievement to the dedication of the authors, the insights of the reviewers, and the ongoing support of the readers. “Everyone's contributions have been crucial to our success, and we are excited to continue our mission to advance knowledge and foster scholarly discourse in the years ahead,” he expressed with gratitude.

The visualization was created using Python, following guidelines established by Setzer and Cui (2022). “One special feature of the graph is the use of the journal's color scheme, which enhances visual harmony, particularly for the cover design,” Cui explained. The data used to calculate the impact factor was sourced from Clarivate (https://clarivate.com/). For those interested in learning more about this data visualization, Zhongmin Cui can be contacted at [email protected].

We also invite you to participate in the annual EM:IP Cover Graphic/Data Visualization Competition. Details for the 2025 competition can be found in this issue. Your entry could be featured on the cover of a future issue! We're eager to receive your feedback and submissions. Please share your thoughts or questions by emailing Yuan-Ling Liaw at [email protected].

This paper addresses some issues concerning the use of current psychometric models for current (and possibly future) technology-based educational testing (as well as most licensure and certification testing). The intent here is to provide a relatively simple overview that addresses important issues, with little explicit intent to argue strenuously for or against the particular uses of technology discussed here.

Automatic item generation may supply many items instantly and efficiently to assessment and learning environments. Yet, the evaluation of item quality persists to be a bottleneck for deploying generated items in learning and assessment settings. In this study, we investigated the utility of using large-language models, specifically Llama 3-8B, for evaluating automatically generated cloze items. The trained large-language model was able to filter out majority of good and bad items accurately. Evaluating items automatically with instruction-tuned LLMs may aid educators and test developers in understanding the quality of items created in an efficient and scalable manner. The item evaluation process with LLMs may also act as an intermediate step between item creation and field testing to reduce the cost and time associated with multiple rounds of revision.

Over our field's 100-year-plus history, standardization has been a central assumption in test theory and practice. The concept's justification turns on leveling the playing field by presenting all examinees with putatively equivalent experiences. Until relatively recently, our field has accepted that justification almost without question. In this article, I present a case for standardization's antithesis, personalization. Interestingly, personalized assessment has important precedents within the measurement community. As intriguing are some of the divergent ways in which personalization might be realized in practice. Those ways, however, suggest a host of serious issues. Despite those issues, both moral obligation and survival imperative counsel persistence in trying to personalize assessment.