{"title":"提高自我报告项目对变化的敏感度:纵向评估中调整和项目制定的作用","authors":"","doi":"10.1016/j.stueduc.2024.101387","DOIUrl":null,"url":null,"abstract":"<div><p>The capability of scales to detect change, i.e. sensitivity to change, is important in longitudinal studies. We present a quantitative study comparing the effects of alignment to course content and item granularity (generic vs. concrete wording) on the capability of different scales to capture changes in ICT self-efficacy. In an evaluation of a course on teaching with ICT, we used ICT self-report questionnaires for pre-service teachers at the beginning and end of a one-semester in a treatment and a control group (<em>N</em><sub>intervention</sub> = 278 vs. <em>N</em><sub>control</sub> = 395). After propensity score matching on the imputed data, we applied descriptive, frequentist, and Bayesian methods to examine differential scale behavior. Results show more pronounced longitudinal changes for scales on instructional ICT self-efficacy, especially for scales formulated for concrete tasks. Our results suggest that better alignment and specific formulation enhance sensitivity to change in self-reports, and relate to scale construction and selection in educational evaluations.</p></div>","PeriodicalId":47539,"journal":{"name":"Studies in Educational Evaluation","volume":null,"pages":null},"PeriodicalIF":2.6000,"publicationDate":"2024-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0191491X2400066X/pdfft?md5=d9a6313cea4b117a726077cb47b51e3c&pid=1-s2.0-S0191491X2400066X-main.pdf","citationCount":"0","resultStr":"{\"title\":\"Improving sensitivity to change in self-report items: The role of alignment and item formulation in a longitudinal evaluation\",\"authors\":\"\",\"doi\":\"10.1016/j.stueduc.2024.101387\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>The capability of scales to detect change, i.e. sensitivity to change, is important in longitudinal studies. We present a quantitative study comparing the effects of alignment to course content and item granularity (generic vs. concrete wording) on the capability of different scales to capture changes in ICT self-efficacy. In an evaluation of a course on teaching with ICT, we used ICT self-report questionnaires for pre-service teachers at the beginning and end of a one-semester in a treatment and a control group (<em>N</em><sub>intervention</sub> = 278 vs. <em>N</em><sub>control</sub> = 395). After propensity score matching on the imputed data, we applied descriptive, frequentist, and Bayesian methods to examine differential scale behavior. Results show more pronounced longitudinal changes for scales on instructional ICT self-efficacy, especially for scales formulated for concrete tasks. Our results suggest that better alignment and specific formulation enhance sensitivity to change in self-reports, and relate to scale construction and selection in educational evaluations.</p></div>\",\"PeriodicalId\":47539,\"journal\":{\"name\":\"Studies in Educational Evaluation\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":2.6000,\"publicationDate\":\"2024-08-05\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.sciencedirect.com/science/article/pii/S0191491X2400066X/pdfft?md5=d9a6313cea4b117a726077cb47b51e3c&pid=1-s2.0-S0191491X2400066X-main.pdf\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Studies in Educational Evaluation\",\"FirstCategoryId\":\"95\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0191491X2400066X\",\"RegionNum\":2,\"RegionCategory\":\"教育学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"EDUCATION & EDUCATIONAL RESEARCH\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Studies in Educational Evaluation","FirstCategoryId":"95","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0191491X2400066X","RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
Improving sensitivity to change in self-report items: The role of alignment and item formulation in a longitudinal evaluation
The capability of scales to detect change, i.e. sensitivity to change, is important in longitudinal studies. We present a quantitative study comparing the effects of alignment to course content and item granularity (generic vs. concrete wording) on the capability of different scales to capture changes in ICT self-efficacy. In an evaluation of a course on teaching with ICT, we used ICT self-report questionnaires for pre-service teachers at the beginning and end of a one-semester in a treatment and a control group (Nintervention = 278 vs. Ncontrol = 395). After propensity score matching on the imputed data, we applied descriptive, frequentist, and Bayesian methods to examine differential scale behavior. Results show more pronounced longitudinal changes for scales on instructional ICT self-efficacy, especially for scales formulated for concrete tasks. Our results suggest that better alignment and specific formulation enhance sensitivity to change in self-reports, and relate to scale construction and selection in educational evaluations.
期刊介绍:
Studies in Educational Evaluation publishes original reports of evaluation studies. Four types of articles are published by the journal: (a) Empirical evaluation studies representing evaluation practice in educational systems around the world; (b) Theoretical reflections and empirical studies related to issues involved in the evaluation of educational programs, educational institutions, educational personnel and student assessment; (c) Articles summarizing the state-of-the-art concerning specific topics in evaluation in general or in a particular country or group of countries; (d) Book reviews and brief abstracts of evaluation studies.