T. Hogan, Marissa DeStefano, Caitlin Gilby, Dana C. Kosman, Joshua Peri
{"title":"Reviewing the Test Reviews: Quality Judgments and Reviewer Agreements in the Mental Measurements Yearbook","authors":"T. Hogan, Marissa DeStefano, Caitlin Gilby, Dana C. Kosman, Joshua Peri","doi":"10.1080/08957347.2021.1890742","DOIUrl":null,"url":null,"abstract":"ABSTRACT Buros’ Mental Measurements Yearbook (MMY) has provided professional reviews of commercially published psychological and educational tests for over 80 years. It serves as a kind of conscience for the testing industry. For a random sample of 50 entries in the 19th MMY (a total of 100 separate reviews) this study determined the level of qualitative judgment rendered by reviewers and the consistency of those independent reviewers in rendering judgments. Judgments of quality distributed themselves almost uniformly from very good to very bad across the 100 reviews. Agreement among reviewers for a given test was positive but relatively weak. We explore implications of the results and suggest follow-up investigations.","PeriodicalId":51609,"journal":{"name":"Applied Measurement in Education","volume":"34 1","pages":"75 - 84"},"PeriodicalIF":1.1000,"publicationDate":"2021-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/08957347.2021.1890742","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Applied Measurement in Education","FirstCategoryId":"95","ListUrlMain":"https://doi.org/10.1080/08957347.2021.1890742","RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0
Abstract
ABSTRACT Buros’ Mental Measurements Yearbook (MMY) has provided professional reviews of commercially published psychological and educational tests for over 80 years. It serves as a kind of conscience for the testing industry. For a random sample of 50 entries in the 19th MMY (a total of 100 separate reviews) this study determined the level of qualitative judgment rendered by reviewers and the consistency of those independent reviewers in rendering judgments. Judgments of quality distributed themselves almost uniformly from very good to very bad across the 100 reviews. Agreement among reviewers for a given test was positive but relatively weak. We explore implications of the results and suggest follow-up investigations.
期刊介绍：
Because interaction between the domains of research and application is critical to the evaluation and improvement of new educational measurement practices, Applied Measurement in Education" prime objective is to improve communication between academicians and practitioners. To help bridge the gap between theory and practice, articles in this journal describe original research studies, innovative strategies for solving educational measurement problems, and integrative reviews of current approaches to contemporary measurement issues. Peer Review Policy: All review papers in this journal have undergone editorial screening and peer review.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
