Educational Measurement-Issues and Practice最新文献

The Computer-based Case Simulations (CCS) component of the United States Medical Licensing Examination (USMLE) Step 3 was developed to assess the decision-making and patient-management skills of physicians. Process data can provide deep insights into examinees’ behavioral processes related to completing the CCS assessment task. In this paper, we utilized process data to evaluate the impact of shortening allotted time limit by rescoring the CCS cases based on process data extracted at various timestamps that represented different percentages of the original allotted case time. It was found that examinees’ performance as well as the correlation between original and newly generated scores both tended to decrease as the timestamp condition became stricter. The impact of shortening allotted time limit was found marginally associated with case difficulties, but strongly dependent on the case time intensity under the original time setting.

Vertical scales are frequently developed using common item nonequivalent group linking. In this design, one can use upper-grade, lower-grade, or mixed-grade common items to estimate the linking constants that underlie the absolute measurement of growth. Using the Rasch model and a dataset from Curriculum Associates’ i-Ready Diagnostic in math in grades 3–7, we demonstrate how grade-to-grade mean differences in mathematics proficiency appear much larger when upper-grade linking items are used instead of lower-grade items, with linkings based on a mixture of items falling in between. We then consider salient properties of the three calibrated scales including invariance of the different sets of common items to student grade and item difficulty reversals. These exploratory analyses suggest that upper-grade common items in vertical scaling are more subject to threats to score comparability across grades, even though these items also tend to imply the most growth.

This contribution to the Special Issue of EM:IP on the topic of The Past, Present and Future of Educational Measurement concentrates on the present and the future and hence focuses on the goal of improving education. The results of meta-analyses were examined, and it was noted that the largest effect sizes were associated with actual use of formative assessments in classroom settings—hence classroom assessment (in contrast with large-scale assessment). The paper describes micro assessment, which focuses on in-classroom forms of measurement, and then expands this assessment approach to focus on frames beyond that in terms of summative end-of-semester tests (macro). This is followed by a description of how these approaches can be combined using a construct map as the basis for developing and using assessments to span across these two levels in terms of the BEAR Assessment System (BAS). Throughout, this is exemplified using an elementary school program designed to teach students about geometry. Finally, a conclusion summarizes the discussion, and also looks to the future where a meso level of use involves end-of-unit tests.

This article describes an amazing development of methods and models supporting educational measurement together with a much slower evolution of theory about how and what students learn and how educational measurement best supports that learning. Told from the perspective of someone who has lived through many of these changes, the article provides background on these developments and insights into challenges and opportunities for future development.

Educational measurement is a social science that requires both qualitative and quantitative competencies. Qualitative competencies in educational measurement include developing and applying theories of learning, designing instruments, and identifying the social, cultural, historical, and political contexts of measurement. Quantitative competencies include statistical inference, computational fluency, and psychometric modeling. I review 12 commentaries authored by past presidents of the National Council on Measurement in Education (NCME) published in a special issue prompting them to reflect on the past, present, and future of educational measurement. I explain how a perspective on both qualitative and quantitative competencies yields common themes across the commentaries. These include the appeal and challenge of personalization, the necessity of contextualization, and the value of communication and collaboration. I conclude that elevation of both qualitative and quantitative competencies underlying educational measurement provides a clearer sense of how NCME can advance its mission, “to advance theory and applications of educational measurement to benefit society.”

Access to admission tests was greatly restricted during the COVID-19 pandemic resulting in widespread adoption of test-optional policies by colleges and universities. Many institutions adopted such policies on an interim or trial basis, as many others signaled the change would be long term. Several Ivy League institutions and selective public flagship universities have returned to requiring test scores from all applicants citing their own research indicating diversity and ensuring academic success of applicants can be best served by inclusion of test scores in the admissions process. This paper reviews recent research on the impact of test-optional policies on score-sending behaviors of applicants and differential outcomes in college and score sending. Ultimately, test-optional policies are neither the panacea for diversity that proponents suggested nor do they result in a decay of academic outcomes that opponents forecast, but they do have consequences, which colleges will need to weigh going forward.

Large language models are increasingly used in educational and psychological measurement activities. Their rapidly evolving sophistication and ability to detect language semantics make them viable tools to supplement subject matter experts and their reviews of large amounts of text statements, such as educational content standards. This paper presents an application of text embeddings to find relationhips between different sets of educational content standards in a content mapping process. Content mapping is routinely used by state education agencies and is often a requirement of the United States Department of Education peer review process. We discuss the educational measurement problem, propose a formal methodology, demonstrate an application of our proposed approach, and provide measures of its accuracy and potential to support real-world activities.

In 2023, a National Council on Measurement in Education Presidential Task Force developed a consensus framework for foundational competencies in educational measurement to guide graduate programs and subsequent professional development. This article elaborates on the social, cultural, historical, and political context subdomain from that framework. A graduate course on the history of testing and testing policy in the United States is proposed to help measurement professionals develop an understanding of historic belief systems and theories of action that affect every aspect of testing applications—definition of constructs, instrument design, respondents’ interactions, interpretations and use of results, and both intended and unintended consequences. Two, accessible, key readings are proposed for each of 14 weeks addressing the following topics: IQ testing and deficit perspectives; special education placements, disproportionality, and accommodations; grade retention and tracking; college admissions testing; standards-based reforms; 1990s performance assessment innovations; NCLB and school accountability; achievement gaps and opportunity to learn; NAEP and international assessments; standard setting and NAEP achievement levels; Common Core State Standards and ESSA; formative assessment and research on learning; culturally responsive assessment.