Educational Measurement-Issues and Practice最新文献

Data visualization is a core tenet of communicating measurement research and outcomes. Measurement professionals utilize data visualization in various phases of research, including exploration and communication. However, data visualization has not received enough attention in the measurement field. While it is true that many measurement graphics are relatively standard, many others are not and there is a wide variety of visualization quality and effectiveness seen in measurement journals. This article provides an overview of the current data visualization trends in measurement and provides some general tips for effective data visualization, with examples. This article is not a comprehensive treatise on data visualization. Therefore, we provide some resources for additional reading. Finally, we call on the measurement community to pay greater attention to the details of data visualization. We also call on measurement training programs to emphasize statistical reasoning through data visualization.

This study employs the 4-parameter logistic item response theory model to account for the unexpected incorrect responses or slipping effects observed in a large-scale Algebra 1 End-of-Course assessment, including several innovative item formats. It investigates whether modeling the misfit at the upper asymptote has any practical impact on the parameter estimates. With a simulation study, it also investigates the amount of bias in the parameter estimates when the slipping effects are ignored. Findings from the empirical data indicate that the impact of ignoring slipping effects is negligible when the abilities are evaluated within the context of classification of students into performance levels; however, it is present toward the extreme ends of ability continuum within the context of individual abilities. Findings from the simulations reveal that when the proportion of items with the slipping effects is small (20%), ignoring misfit does not have practical importance; however, when the proportion of items with the slipping effects is moderate to large (50%–80%), the abilities are generally underestimated at both ends of ability scale. When an upper asymptote parameter was used for modeling the slipping effects, the items became easier and more discriminative in general than the model ignoring the slipping effects.

COVID-19 is disrupting assessment practices and accelerating changes. With special focus on K-12 and credentialing exams, this article describes the series of changes observed during the pandemic, the solutions assessment providers have implemented, and the long-term impact on future practices. Additionally, this article highlights the importance of the balanced assessment system, the use of assessments both for learning and of learning, and using assessments to support for social justice, equity and inclusion. These desired uses and outcomes will continue to challenge assessment and measurement experts on how they design, develop, and implement assessments moving forward.

Test equating requires collecting data to link the scores from different forms of a test. Problems arise when equating samples are not equivalent and the test forms to be linked share no common items by which to measure or adjust for the group nonequivalence. Using data from five operational test forms, we created five pairs of research forms for each form, such that the equating relationship between each pair was known. Then we compared five approaches to adjusting for group nonequivalence in a situation where not only was group equivalence questionable, but the number of common items was small. We used a resampling approach to evaluate the linking accuracy of group adjustment using sample weights via minimum discriminant information adjustment (MDIA) using test takers’ collateral (demographic) information, a weak anchor of only three items, or a mix of both. Overall, the use of both sample weights via MDIA and a weak anchor produced the most accurate result, while the direct (random groups) linking method assuming group equivalence produced the least accurate result due to nontrivial bias. For all five research forms, using both collateral information and anchor items only marginally improved linking accuracy compared to using the weak anchor alone.

Formal graduate education in a measurement related field provides a solid foundation for professionals who work on credentialing examinations. Those foundational skills are then expanded and refined over time as practitioners encounter complex and nuanced challenges that were not covered by or go beyond the context described in textbooks. For instance, as most of us who work on operational testing programs are (sometimes) painfully aware, real data can be very messy. Often unanticipated situations arise that can create a range of problems from threats to score validity, to unexpected financial costs, and even longer-term reputational damage. In practice, solutions for these unanticipated situations are not always straightforward, often requiring a compromise between psychometric best practices, business resources, and needs of the customer. In this module we discuss some of these unusual challenges that usually occur in a credentialing program. First, we provide a high-level summary of the main components of the assessment lifecycle and the different roles within a testing organization. Next, we propose a framework for qualifying risk along with various considerations and potential actions for managing these challenges. Lastly, we integrate this information by presenting a few scenarios that can occur in practice that should help learners think though applicable team-based problem-solving and better align recommended action from a psychometric perspective given the context and magnitude of the challenge.

Welcome to the first issue of the Instructional Topics in Educational Measurement Series (ITEMS) of my 3-year tenure as editor. I first want to thank André Rupp, the outgoing editor, for his patience and time discussing technical details of the modern ITEMS module format (more on his work below). I also want to thank Susan Davis-Becker and Michael Peabody for their work leading the Publications Committee and guidance through the transition process. Finally, I am thankful for the assistance from Deborah Harris, outgoing editor of Educational Measurement: Issues and Practice (EM:IP), and Zhongmin Cui, incoming editor, with whom I look forward to working over our tenures.

I am grateful for the opportunity to lead ITEMS. As an academic, I expend considerable energy teaching future professionals of our field. However, being an educator is much more than simply teaching. It also requires advising, mentorship, and recruiting. I envision ITEMS as a confluence of these on a grand scale. Of course, teaching and learning is at the core of ITEMS. However, informing others about educational measurement involves more than teaching a technical short course on item response theory or reliability. It also requires catering to the needs of the learner, providing resources to fill gaps in knowledge or to review material learned years ago, as well as giving guidance to young professionals about the opportunities in the field.

Since 1987, ITEMS has been published as a part of EM:IP. Until May 2017, ITEMS modules were published as didactic articles, appealing to practitioners seeking more technical expertise, K–12 teachers desiring resources to learn about assessment and measurement, as well as graduate students desiring to supplement their education. After a hiatus of just over a year, ITEMS returned in 2018 in a new form. To bring ITEMS into the 21st century (Rupp, 2018), then-editor André Rupp reimagined ITEMS for a digital landscape. Over the next three and a half years twenty-seven high-quality digital modules were published. These digital modules blended voice-narrated content with applied examples, exercise sets, software exemplars, and other unique resources only possible on a digital platform. These modules would not have been possible without the expertise and dedication of the authors developing the incredible products. The behind-the-scenes instructional design team also deserves recognition. These volunteers worked countless hours to implement the vision of what we now see as digital ITEMS modules. Having learned the intricacies of building a module, it is abundantly clear that these digital modules would not have been possible without the instructional team's effort. Thank you, Xi Lu and Jonathan Lehrfeld for your substantial contribution.

As I begin my tenure, I plan to build on the accomplished history of ITEMS with new modules appealing to the varied interests both within and beyond the NCME commu