The purpose of this study was to compare the results and interpretation of the data from a performance examination when four methods of analysis are used. Methods are 1) traditional summary statistics, 2) inter-judge correlations, 3) generalizability theory, and 4) the multi-facet Rasch model. Results indicated that similar sources of variance were identified using each method; however, the multi-facet Rasch model is the only method that linearized the scores and accounts for differences in the particular examination challenged by a candidate before ability estimates are calculated.
{"title":"Scoring and analysis of performance examinations: a comparison of methods and interpretations.","authors":"M E Lunz, R E Schumacker","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>The purpose of this study was to compare the results and interpretation of the data from a performance examination when four methods of analysis are used. Methods are 1) traditional summary statistics, 2) inter-judge correlations, 3) generalizability theory, and 4) the multi-facet Rasch model. Results indicated that similar sources of variance were identified using each method; however, the multi-facet Rasch model is the only method that linearized the scores and accounts for differences in the particular examination challenged by a candidate before ability estimates are calculated.</p>","PeriodicalId":79673,"journal":{"name":"Journal of outcome measurement","volume":"1 3","pages":"219-38"},"PeriodicalIF":0.0,"publicationDate":"1997-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"20579310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
It is often impossible to validate cut scores set using judged item review methods due to the fact that many high stakes testing programs attempt to limit the number of common items across consecutively administered forms. However, over time, with a stable item pool, secondary links through other test administrations allow the use of common item equating to test the stability of the judged cut scores. In this study five forms of a basic science examination administered over a three year period in a national board testing program were analyzed to determine the stability of judged cut scores. The stability was determined by comparison of the judged cut scores with the equated cut scores derived by the Rasch common item equating technique. The results indicate cut scores derived from the modified Nedelsky procedure were within equating error of the Rasch equated cut scores over five administrations.
{"title":"Validating standard setting with a modified nedelsky procedure through common item test equating.","authors":"R M Smith, L J Gross","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>It is often impossible to validate cut scores set using judged item review methods due to the fact that many high stakes testing programs attempt to limit the number of common items across consecutively administered forms. However, over time, with a stable item pool, secondary links through other test administrations allow the use of common item equating to test the stability of the judged cut scores. In this study five forms of a basic science examination administered over a three year period in a national board testing program were analyzed to determine the stability of judged cut scores. The stability was determined by comparison of the judged cut scores with the equated cut scores derived by the Rasch common item equating technique. The results indicate cut scores derived from the modified Nedelsky procedure were within equating error of the Rasch equated cut scores over five administrations.</p>","PeriodicalId":79673,"journal":{"name":"Journal of outcome measurement","volume":"1 2","pages":"164-72"},"PeriodicalIF":0.0,"publicationDate":"1997-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"20579307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号