Educational Measurement-Issues and Practice最新文献

What are foundational competencies in educational measurement? We published a framework for these foundational competencies in this journal (Ackerman et al. 2024) and were grateful to receive eight commentaries raising a number of important questions about the framework and its implications. We identified five cross-cutting recommendations among the eight commentaries relating to (1) our process and purpose, (2) Artificial Intelligence, (3) ethical competencies, (4) qualitative, critical, and culturally responsive commentaries, and (5) intersecting professions in, for example, classroom assessment and content development. In this rejoinder, we respond to these five recommendations and to each of the eight commentaries in turn. We hope that discussion and consensus about foundational competencies in educational measurement continue to advance in our journals and our field.

The “Foundational Competencies in Educational Measurement” framework aims to shape the field's future. However, the absence of emerging scholars and graduate students in the task force highlights a gap in representing those most familiar with current educational landscape. As early career scholars, we offer perspectives to enhance this framework by focusing on making educational measurement more inclusive, collaborative, and culturally responsive. Drawing on our diverse backgrounds and experiences, we propose expanding the framework to empower measurement professionals, diversify measurement practices, and integrate ethical considerations. We also advocate for a new visual representation of the framework as a budding plant, symbolizing the organic and evolving nature of foundational skills in educational measurement. This commentary aims to refine the foundational competencies to better prepare future professionals for meaningful, equitable educational contributions.

There is a growing integration of technology in the workplace (World Economic Forum), and with it, organizations are increasingly relying on advanced technological approaches for improving their human capital processes to stay relevant and competitive in complex environments. All professions must keep up with this transition and begin integrating technology into their tools and processes. This paper centers on how advanced technological approaches (such as natural language processing (NLP) and data mining) have complemented a traditional practice analysis of the accounting profession. We also discuss strategic selection and use of subject-matter experts (SMEs) for more efficient practice analysis. The authors have adopted a triangulation process—gathering information from traditional practice analysis, using selected SMEs, and confirming findings with a novel NLP-based approach. These methods collectively contributed to the revision of the Uniform CPA Exam blueprint and in understanding accounting trends.

This article is based on my 2023 NCME Presidential Address, where I talked a bit about my journey into the profession, and more substantively about comparable scores. Specifically, I discussed some of the different ways ‘comparable scores’ are defined, highlighted some areas I think we as a profession need to pay more attention to when considering score comparability, and emphasized that comparability in this context is a matter of degree which varies according to the decisions we plan to make on particular scores.

This study capitalizes on response and process data from the computer-based TIMSS 2019 Problem Solving and Inquiry tasks to investigate gender differences in test-taking behaviors and their association with mathematics achievement at the eighth grade. Specifically, a recently proposed hierarchical speed-accuracy-revisits (SAR) model was adapted to multiple country-by-gender groups to examine the extent to which mathematics ability, response speed, revisit propensity, and the relationship among them differ between boys and girls. Results across 10 countries showed that boys responded to items faster on average than girls, and there was greater variation in boys’ response speed across students. A mixture distribution of revisit propensity was found for all country-by-gender groups. Both genders had moderate to strong negative correlations between mathematics ability and response speed, supporting the speed-accuracy tradeoff pattern reported in the literature. Results are discussed in the context of low-stakes assessments and in relation to the utility of the multiple-group SAR model.

Traditional approaches to the modeling of multiple-choice item response data (e.g., 3PL, 4PL models) emphasize slips and guesses as random events. In this paper, an item response model is presented that characterizes both disjunctively interacting guessing and conjunctively interacting slipping processes as proficiency-related phenomena. We show how evidence for this perspective is seen in the systematic form of invariance violations for item slip and guess parameters under four-parameter IRT models when compared across populations of different mean proficiency levels. Specifically, higher proficiency populations tend to show higher guess and lower slip probabilities than lower proficiency populations. The results undermine the use of traditional models for IRT applications that require invariance and would suggest greater attention to alternatives.