Pub Date : 1900-01-01DOI: 10.20622/jltajournal.23.0_37
This study is aimed at developing a Japanese listening test for cognitive diagnostic assessment at the CEFR B1 level. The paper describes improvements to the attributes and test items based on the values of guessing and slip parameters estimated by the DINA model and the retrospective verbal report method, and discusses whether these methods are effective for developing cognitive diagnostic assessments. First, CEFR materials were analyzed, and nine attributes were identified. Next, a Japanese listening test consisting of 27 items measuring these attributes was developed, and a 12 × 27 Q-matrix was constructed. Two surveys were then conducted. Survey 1 obtained answers to the test from 297 learners of the Japanese language. Survey 2 obtained retrospective verbal reports from 13 learners to understand the answering process. The test items were then analyzed based on the estimated parameter values and the findings from the retrospective verbal reports. The attributes were modified, some of the test items were eliminated, and the Q-matrix was reconstructed. When re-estimation was performed after revision, the parameter values were improved. It was confirmed that the interpretation conducted by the retrospective verbal reports was valid and the effectiveness of our methods for developing a new cognitive diagnostic assessment was demonstrated.
{"title":"Analysis of Attributes and Items for Developing a Cognitive Diagnostic Japanese Listening Test","authors":"","doi":"10.20622/jltajournal.23.0_37","DOIUrl":"https://doi.org/10.20622/jltajournal.23.0_37","url":null,"abstract":"This study is aimed at developing a Japanese listening test for cognitive diagnostic assessment at the CEFR B1 level. The paper describes improvements to the attributes and test items based on the values of guessing and slip parameters estimated by the DINA model and the retrospective verbal report method, and discusses whether these methods are effective for developing cognitive diagnostic assessments. First, CEFR materials were analyzed, and nine attributes were identified. Next, a Japanese listening test consisting of 27 items measuring these attributes was developed, and a 12 × 27 Q-matrix was constructed. Two surveys were then conducted. Survey 1 obtained answers to the test from 297 learners of the Japanese language. Survey 2 obtained retrospective verbal reports from 13 learners to understand the answering process. The test items were then analyzed based on the estimated parameter values and the findings from the retrospective verbal reports. The attributes were modified, some of the test items were eliminated, and the Q-matrix was reconstructed. When re-estimation was performed after revision, the parameter values were improved. It was confirmed that the interpretation conducted by the retrospective verbal reports was valid and the effectiveness of our methods for developing a new cognitive diagnostic assessment was demonstrated.","PeriodicalId":249185,"journal":{"name":"JLTA Journal","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128419390","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：