首页 > 最新文献

Journal of Measurement and Evaluation in Education and Psychology-EPOD最新文献

英文 中文
Design and Development of an Interactive Video Player for Supporting Formative Assessment in Online Learning 支持在线学习形成性评价的交互式视频播放器的设计与开发
Q4 Social Sciences Pub Date : 2023-09-10 DOI: 10.21031/epod.1286077
Alper BAYAZIT, Gökhan AKÇAPINAR
In this study, the aim was to design a video player with embedded quizzes to enhance students' interactions with video learning materials and enable them to derive maximum benefit from these materials. The developed tool is integrated into the Moodle learning management system and presents questions from a question bank to students at predetermined time intervals set by the instructor. Additionally, it records interactions between the student and embedded quizzes, such as the number of attempts and the number of correct or incorrect answers. Furthermore, students' video interactions, encompassing actions like play, pause, and seek, are also stored in the database. The tool allows instructors to export 15 features related to students' videos and embedded quiz interactions. Consequently, students can assess their comprehension of the content they are viewing and receive immediate feedback, while instructors can access summary reports for all students. This enables them to find out how many students have watched the videos and their responses to the questions before or after the class. This paper explains the development process of the tool and presents findings from a pilot study utilizing the tool. The results of the pilot study revealed that students' video-watching behaviors in interactive videos differed from those in non-interactive videos. An interactive video player that allows embedding quiz questions to support formative assessment in online learning environments, can be advantageous for researchers, instructors, and learners.
在这项研究中,目的是设计一个嵌入测验的视频播放器,以增强学生与视频学习材料的互动,使他们能够从这些材料中获得最大的收益。开发的工具集成到Moodle学习管理系统中,并在教师设定的预定时间间隔内向学生提供题库中的问题。此外,它还记录学生和嵌入式测验之间的交互,例如尝试次数和正确或错误答案的数量。此外,学生的视频互动,包括播放、暂停和寻找等动作,也存储在数据库中。该工具允许教师导出15个与学生视频和嵌入式测验互动相关的功能。因此,学生可以评估他们对所看内容的理解程度并获得即时反馈,而教师可以访问所有学生的总结报告。这使他们能够了解有多少学生观看了视频以及他们在课前或课后对问题的回答。本文解释了该工具的开发过程,并介绍了利用该工具进行的初步研究的结果。初步研究结果表明,学生在互动视频中观看视频的行为与在非互动视频中观看视频的行为不同。交互式视频播放器允许嵌入测验问题,以支持在线学习环境中的形成性评估,这对研究人员、教师和学习者都是有利的。
{"title":"Design and Development of an Interactive Video Player for Supporting Formative Assessment in Online Learning","authors":"Alper BAYAZIT, Gökhan AKÇAPINAR","doi":"10.21031/epod.1286077","DOIUrl":"https://doi.org/10.21031/epod.1286077","url":null,"abstract":"In this study, the aim was to design a video player with embedded quizzes to enhance students' interactions with video learning materials and enable them to derive maximum benefit from these materials. The developed tool is integrated into the Moodle learning management system and presents questions from a question bank to students at predetermined time intervals set by the instructor. Additionally, it records interactions between the student and embedded quizzes, such as the number of attempts and the number of correct or incorrect answers. Furthermore, students' video interactions, encompassing actions like play, pause, and seek, are also stored in the database. The tool allows instructors to export 15 features related to students' videos and embedded quiz interactions. Consequently, students can assess their comprehension of the content they are viewing and receive immediate feedback, while instructors can access summary reports for all students. This enables them to find out how many students have watched the videos and their responses to the questions before or after the class. This paper explains the development process of the tool and presents findings from a pilot study utilizing the tool. The results of the pilot study revealed that students' video-watching behaviors in interactive videos differed from those in non-interactive videos. An interactive video player that allows embedding quiz questions to support formative assessment in online learning environments, can be advantageous for researchers, instructors, and learners.","PeriodicalId":43015,"journal":{"name":"Journal of Measurement and Evaluation in Education and Psychology-EPOD","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136108526","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Examining Students' Formative Test-Taking Behaviors Using Learning Analytics 运用学习分析研究学生的形成性应试行为
Q4 Social Sciences Pub Date : 2023-09-03 DOI: 10.21031/epod.1275597
Alper BAYAZIT, Denizer YILDIRIM, Gökhan AKÇAPINAR, Hale ILGAZ
In online learning environments, assessment is an important dimension and also one of the most challenging parts of the process. So to provide an effective learning process analyzing students’ behaviors are important for designing formative and summative assessment environments. In this study, students’ profiles have been analyzed in a formative assessment environment and compared with the summative assessment environment based on attempt count, overall time spent, first attempt score, and the last attempt score metrics. The within-subjects design has been used. Cluster analysis and the Kruskal Wallis-H Test were used to analyze behaviors. The data show that there are three main clusters. Cluster 1 showed a high number of interactions, and an increasing trend was observed in grades over attempts. Cluster 2 consists of the students who received the best grades in all of the other clusters, and lastly, cluster 3 consists of students that interact little and score lower on formative assessments.
在在线学习环境中,评估是一个重要的方面,也是这个过程中最具挑战性的部分之一。因此,提供一个有效的学习过程,分析学生的行为对设计形成性和总结性评价环境具有重要意义。在本研究中,学生在形成性评估环境中进行了分析,并根据尝试次数、总花费时间、第一次尝试分数和最后一次尝试分数指标与总结性评估环境进行了比较。采用了主题内设计。采用聚类分析和Kruskal Wallis-H检验对行为进行分析。数据显示,主要有三个集群。集群1显示了大量的相互作用,并且在尝试中观察到分数的增加趋势。第二组由在其他所有组中成绩最好的学生组成,最后,第三组由互动很少且在形成性评估中得分较低的学生组成。
{"title":"Examining Students' Formative Test-Taking Behaviors Using Learning Analytics","authors":"Alper BAYAZIT, Denizer YILDIRIM, Gökhan AKÇAPINAR, Hale ILGAZ","doi":"10.21031/epod.1275597","DOIUrl":"https://doi.org/10.21031/epod.1275597","url":null,"abstract":"In online learning environments, assessment is an important dimension and also one of the most challenging parts of the process. So to provide an effective learning process analyzing students’ behaviors are important for designing formative and summative assessment environments. In this study, students’ profiles have been analyzed in a formative assessment environment and compared with the summative assessment environment based on attempt count, overall time spent, first attempt score, and the last attempt score metrics. The within-subjects design has been used. Cluster analysis and the Kruskal Wallis-H Test were used to analyze behaviors. The data show that there are three main clusters. Cluster 1 showed a high number of interactions, and an increasing trend was observed in grades over attempts. Cluster 2 consists of the students who received the best grades in all of the other clusters, and lastly, cluster 3 consists of students that interact little and score lower on formative assessments.","PeriodicalId":43015,"journal":{"name":"Journal of Measurement and Evaluation in Education and Psychology-EPOD","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134949484","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
THE ROLE OF EPORTFOLIOS IN FORMATIVE ASSESSMENT: A SYSTEMATIC LITERATURE REVIEW 电子档案在形成性评估中的作用:系统的文献回顾
Q4 Social Sciences Pub Date : 2023-08-28 DOI: 10.21031/epod.1276063
Ezgi GÜN-TOSİK, Bilal ATASOY, Akça Okan YÜKSEL
In educational settings, formative assessment is used to determine the status of students and foster their development. Task, process, and feedback serve as the basis of formative assessments. Typically, tasks are introduced to lessons to facilitate student feedback. Due to their technological potential, eportfolios are often used to help with assigning tasks, managing processes, and getting feedback. It is necessary to incorporate the findings from environments where eportfolios are used for formative assessment into the existing literature. A systematic literature review was employed for this purpose. Three researchers reviewed 33 studies utilizing content analysis. The Fleiss kappa was used to determine inter-rater reliability. Quantitative analysis was performed on the articles' dependent or related variables, environments, research method, implementation period, evaluators, eportfolio type, activity format, evaluation instruments, education level, and course categories. Among the qualitative findings of the studies were the prominent concepts of the use of eportfolios, as well as the themes and codes of advantages and challenges. The findings indicate that the integration of eportfolios in the formative assessment process, despite its challenges, provides significant advantages. It is expected that the study's findings will be useful for researchers as well as practitioners who intend to use eportfolios for formative assessment processes.
在教育环境中,形成性评价被用来确定学生的状况并促进他们的发展。任务、过程和反馈是形成性评估的基础。通常,在课程中引入任务是为了促进学生的反馈。由于它们的技术潜力,电子投资组合经常被用来帮助分配任务、管理过程和获得反馈。有必要将电子档案袋用于形成性评估的环境中的发现纳入现有文献。为此目的采用了系统的文献综述。三位研究人员利用内容分析回顾了33项研究。采用Fleiss kappa法测定量表间信度。定量分析文章的因变量或相关变量、环境、研究方法、实施周期、评估者、项目类型、活动形式、评估工具、教育水平和课程类别。这些研究的定性发现包括使用电子投资组合的突出概念,以及优势和挑战的主题和准则。研究结果表明,在形成性评价过程中整合电子作品集尽管存在挑战,但具有显著的优势。预计该研究的结果将对打算使用电子档案进行形成性评估过程的研究人员和从业者有用。
{"title":"THE ROLE OF EPORTFOLIOS IN FORMATIVE ASSESSMENT: A SYSTEMATIC LITERATURE REVIEW","authors":"Ezgi GÜN-TOSİK, Bilal ATASOY, Akça Okan YÜKSEL","doi":"10.21031/epod.1276063","DOIUrl":"https://doi.org/10.21031/epod.1276063","url":null,"abstract":"In educational settings, formative assessment is used to determine the status of students and foster their development. Task, process, and feedback serve as the basis of formative assessments. Typically, tasks are introduced to lessons to facilitate student feedback. Due to their technological potential, eportfolios are often used to help with assigning tasks, managing processes, and getting feedback. It is necessary to incorporate the findings from environments where eportfolios are used for formative assessment into the existing literature. A systematic literature review was employed for this purpose. Three researchers reviewed 33 studies utilizing content analysis. The Fleiss kappa was used to determine inter-rater reliability. Quantitative analysis was performed on the articles' dependent or related variables, environments, research method, implementation period, evaluators, eportfolio type, activity format, evaluation instruments, education level, and course categories. Among the qualitative findings of the studies were the prominent concepts of the use of eportfolios, as well as the themes and codes of advantages and challenges. The findings indicate that the integration of eportfolios in the formative assessment process, despite its challenges, provides significant advantages. It is expected that the study's findings will be useful for researchers as well as practitioners who intend to use eportfolios for formative assessment processes.","PeriodicalId":43015,"journal":{"name":"Journal of Measurement and Evaluation in Education and Psychology-EPOD","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-08-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135135131","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
THE EFFECT OF USING E-PORTFOLIOS ON THE SELF-REGULATION SKILLS OF STUDENTS: A META-ANALYSIS STUDY 使用电子档案对学生自我调节技能的影响:一项元分析研究
Q4 Social Sciences Pub Date : 2023-06-20 DOI: 10.21031/epod.1267809
Ceren MUTLUER
The change of self-regulation skills, one of the variables determining success in education, with measurement and evaluation techniques in the education process has been the subject of many studies. However, each research result leads to different results due to the planned situation and limitations. For this reason, e-portfolios used in the education process were examined with a meta-analysis study to evaluate whether they were effective on students' self-regulation skills from a more general point of view. The research is limited to published academic studies written in Turkish and English between 2000 and 2023. When the criteria determined in the research were examined, only 19 studies were found by these criteria. Theoretically, when the effect sizes in each study were examined, a meta-analysis was carried out with the random effects model. The analyzes of the research were made with the CMA version 3 program. As a result of the research, it was concluded that using e-portfolios greatly and significantly affect students' self-regulation skills.
作为决定教育成功的变量之一,自我调节技能在教育过程中的变化与测量和评估技术一直是许多研究的主题。然而,由于计划的情况和局限性,每个研究结果都会导致不同的结果。出于这个原因,在教育过程中使用的电子档案通过一项荟萃分析研究来评估它们是否从更普遍的角度对学生的自我调节技能有效。这项研究仅限于2000年至2023年间用土耳其语和英语撰写的已发表的学术研究。当检查研究中确定的标准时,只有19项研究符合这些标准。理论上,在检验每项研究的效应量时,采用随机效应模型进行meta分析。利用CMA第三版程序对研究结果进行了分析。研究结果表明,电子档案的使用对学生的自我调节能力有显著的影响。
{"title":"THE EFFECT OF USING E-PORTFOLIOS ON THE SELF-REGULATION SKILLS OF STUDENTS: A META-ANALYSIS STUDY","authors":"Ceren MUTLUER","doi":"10.21031/epod.1267809","DOIUrl":"https://doi.org/10.21031/epod.1267809","url":null,"abstract":"The change of self-regulation skills, one of the variables determining success in education, with measurement and evaluation techniques in the education process has been the subject of many studies. However, each research result leads to different results due to the planned situation and limitations. For this reason, e-portfolios used in the education process were examined with a meta-analysis study to evaluate whether they were effective on students' self-regulation skills from a more general point of view. The research is limited to published academic studies written in Turkish and English between 2000 and 2023. When the criteria determined in the research were examined, only 19 studies were found by these criteria. Theoretically, when the effect sizes in each study were examined, a meta-analysis was carried out with the random effects model. The analyzes of the research were made with the CMA version 3 program. As a result of the research, it was concluded that using e-portfolios greatly and significantly affect students' self-regulation skills.","PeriodicalId":43015,"journal":{"name":"Journal of Measurement and Evaluation in Education and Psychology-EPOD","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135288193","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Analysis of Factors Affecting Individuals' Sources of Happiness with Multinomial Logistic Model 多元Logistic模型分析影响个体幸福来源的因素
IF 0.5 Q4 Social Sciences Pub Date : 2021-09-29 DOI: 10.21031/EPOD.925631
Kübranur Çebi Karaaslan
The happiness levels of individuals and their sources of happiness have been wondered a lot and researched from past to present. The aim of this study is to examine the factors that affect individuals' sources of happiness. The data set of the study was obtained from the Life Satisfaction Survey of the Turkish Statistical Institute. 9212 individuals were included in the study. In the study, chi-square independence tests were conducted to examine the relationship between the source of happiness and the independent variables included in the model, and multinomial logistic regression analysis was applied to determine the factors that may have an effect on the sources of happiness of individuals. As a result of the study, it has been determined that the factors of the individual's age, gender, marital status, educational status, satisfaction with income level, welfare level, life satisfaction, satisfaction with a social life are effective on sources of happiness. At such a time when it is clear that the coronavirus epidemic adversely affects many aspects of our lives, especially our psychology, and will leave a mark on our tomorrows, and the activities of decision-makers and policymakers are shed light through the study in order to increase the happiness of individuals and to ensure that the future will be better.
从过去到现在,人们对个人的幸福水平及其幸福来源进行了大量的猜测和研究。这项研究的目的是检验影响个人幸福来源的因素。该研究的数据集来自土耳其统计研究所的生活满意度调查。9212人被纳入研究。在这项研究中,采用卡方独立性检验来检验幸福感来源与模型中包含的自变量之间的关系,并采用多项逻辑回归分析来确定可能影响个人幸福感来源的因素。研究结果表明,个体的年龄、性别、婚姻状况、教育状况、对收入水平的满意度、福利水平、生活满意度、对社会生活的满意度等因素对幸福感的来源是有效的。在这样一个时刻,冠状病毒疫情显然对我们生活的许多方面,特别是我们的心理产生了不利影响,并将在我们的明天留下印记,决策者和决策者的活动通过这项研究得以揭示,以增加个人的幸福感,并确保未来会更好。
{"title":"Analysis of Factors Affecting Individuals' Sources of Happiness with Multinomial Logistic Model","authors":"Kübranur Çebi Karaaslan","doi":"10.21031/EPOD.925631","DOIUrl":"https://doi.org/10.21031/EPOD.925631","url":null,"abstract":"The happiness levels of individuals and their sources of happiness have been wondered a lot and researched from past to present. The aim of this study is to examine the factors that affect individuals' sources of happiness. The data set of the study was obtained from the Life Satisfaction Survey of the Turkish Statistical Institute. 9212 individuals were included in the study. In the study, chi-square independence tests were conducted to examine the relationship between the source of happiness and the independent variables included in the model, and multinomial logistic regression analysis was applied to determine the factors that may have an effect on the sources of happiness of individuals. As a result of the study, it has been determined that the factors of the individual's age, gender, marital status, educational status, satisfaction with income level, welfare level, life satisfaction, satisfaction with a social life are effective on sources of happiness. At such a time when it is clear that the coronavirus epidemic adversely affects many aspects of our lives, especially our psychology, and will leave a mark on our tomorrows, and the activities of decision-makers and policymakers are shed light through the study in order to increase the happiness of individuals and to ensure that the future will be better.","PeriodicalId":43015,"journal":{"name":"Journal of Measurement and Evaluation in Education and Psychology-EPOD","volume":null,"pages":null},"PeriodicalIF":0.5,"publicationDate":"2021-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47285102","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Comparison of Testlet Effect on Parameter Estimates Using Different Item Response Theory Models 不同项目反应理论模型对参数估计的测试集效应比较
IF 0.5 Q4 Social Sciences Pub Date : 2021-09-27 DOI: 10.21031/EPOD.948227
E. Koğar
{"title":"Comparison of Testlet Effect on Parameter Estimates Using Different Item Response Theory Models","authors":"E. Koğar","doi":"10.21031/EPOD.948227","DOIUrl":"https://doi.org/10.21031/EPOD.948227","url":null,"abstract":"","PeriodicalId":43015,"journal":{"name":"Journal of Measurement and Evaluation in Education and Psychology-EPOD","volume":null,"pages":null},"PeriodicalIF":0.5,"publicationDate":"2021-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44514036","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Investigation of Classification Accuracy, Test Length and Measurement Precision at Computerized Adaptive Classification Tests 计算机自适应分类测试中分类精度、测试长度和测量精度的研究
IF 0.5 Q4 Social Sciences Pub Date : 2021-02-21 DOI: 10.21031/EPOD.787865
Sedanur Demir, B. Atar
This study aims to compare Sequential Probability Ratio Test (SPRT) and Confidence Interval (CI) classification criteria, Maximum Fisher Information method on the basis of estimated-ability (MFI-EB) and Cut-Point (MFI-CB) item selection methods while ability estimation method is Weighted Likelihood Estimation (WLE) in Computerized Adaptive Classification Testing (CACT), according to the Average Classification Accuracy (ACA), Average Test Length (ATL), and measurement precision under content balancing (Constrained Computerized Adaptive Testing: CCAT and Modified Multinomial Model: MMM) and item exposure control (Sympson-Hetter Method: SH and Item Eligibility Method: IE) when the classification is done based on two, three, or four categories for a unidimensional pool of dichotomous items. Forty-eight conditions are created in Monte Carlo (MC) simulation for the data, generated in R software, including 500 items and 5000 examinees, and the results are calculated over 30 replications. As a result of the study, it was observed that CI performs better in terms of ATL, and SPRT performs better in ACA and correlation, bias, Root Mean Squared Error (RMSE), and Mean Absolute Error (MAE) values, sequentially; MFI-EB is more useful than MFI-CB. It was also seen that MMM is more successful in content balancing, whereas CCAT is better in terms of test efficiency (ATL and ACA), and IE is superior in terms of item exposure control though SH is more beneficial in test efficiency. Besides, increasing the number of classification categories increases ATL but decreases ACA, and it gives better results in terms of the correlation, bias, RMSE, and MAE values.
本研究旨在比较计算机自适应分类测试(CACT)中能力估计方法为加权似然估计(WLE)的顺序概率比检验(SPRT)和置信区间(CI)分类标准、基于估计能力的最大Fisher信息法(MFI-EB)和Cut-Point (MFI-CB)项目选择方法,根据平均分类准确率(ACA)、平均测试长度(ATL)、在内容平衡(约束计算机自适应测试:CCAT和修正多项模型:MMM)和项目暴露控制(simpson - hetter方法:SH和项目合格性方法:IE)下,当分类基于两个、三个或四个类别对一维二分类项目进行分类时,测量精度。在蒙特卡罗(MC)模拟中,对R软件生成的数据创建了48个条件,包括500个项目和5000名考生,并对结果进行了30多次重复计算。研究结果显示,CI在ATL方面表现较好,SPRT在ACA和相关性、偏倚、均方根误差(RMSE)和平均绝对误差(MAE)值方面表现较好;MFI-EB比MFI-CB更有用。MMM在内容平衡方面更成功,而CCAT在测试效率(ATL和ACA)方面更好,IE在项目暴露控制方面更优越,而SH在测试效率方面更有利。此外,增加分类类别的数量会增加ATL而降低ACA,并且在相关性、偏倚、RMSE和MAE值方面都能得到更好的结果。
{"title":"Investigation of Classification Accuracy, Test Length and Measurement Precision at Computerized Adaptive Classification Tests","authors":"Sedanur Demir, B. Atar","doi":"10.21031/EPOD.787865","DOIUrl":"https://doi.org/10.21031/EPOD.787865","url":null,"abstract":"This study aims to compare Sequential Probability Ratio Test (SPRT) and Confidence Interval (CI) classification criteria, Maximum Fisher Information method on the basis of estimated-ability (MFI-EB) and Cut-Point (MFI-CB) item selection methods while ability estimation method is Weighted Likelihood Estimation (WLE) in Computerized Adaptive Classification Testing (CACT), according to the Average Classification Accuracy (ACA), Average Test Length (ATL), and measurement precision under content balancing (Constrained Computerized Adaptive Testing: CCAT and Modified Multinomial Model: MMM) and item exposure control (Sympson-Hetter Method: SH and Item Eligibility Method: IE) when the classification is done based on two, three, or four categories for a unidimensional pool of dichotomous items. Forty-eight conditions are created in Monte Carlo (MC) simulation for the data, generated in R software, including 500 items and 5000 examinees, and the results are calculated over 30 replications. As a result of the study, it was observed that CI performs better in terms of ATL, and SPRT performs better in ACA and correlation, bias, Root Mean Squared Error (RMSE), and Mean Absolute Error (MAE) values, sequentially; MFI-EB is more useful than MFI-CB. It was also seen that MMM is more successful in content balancing, whereas CCAT is better in terms of test efficiency (ATL and ACA), and IE is superior in terms of item exposure control though SH is more beneficial in test efficiency. Besides, increasing the number of classification categories increases ATL but decreases ACA, and it gives better results in terms of the correlation, bias, RMSE, and MAE values.","PeriodicalId":43015,"journal":{"name":"Journal of Measurement and Evaluation in Education and Psychology-EPOD","volume":null,"pages":null},"PeriodicalIF":0.5,"publicationDate":"2021-02-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48725532","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Silent Predictors of Test Disengagement in PIAAC 2012 PIAAC 2012中测试脱离的沉默预测因子
IF 0.5 Q4 Social Sciences Pub Date : 2020-12-13 DOI: 10.21031/epod.796626
Münevver İLGÜN DİBEK
Although the effects of test disengagement on the validity of the scores obtained from the data set have been examined in many studies, the predictors of the disengaged behaviors received relatively limited scholarly attention in low-stakes assessment, in particular, in international comparison studies. As such, the present study with a twofold purpose sets out to determine the best fitted explanatory item response theory model and examine the predictors of test disengagement. The data were collected by using items measuring literacy and numeracy skills of adults from different countries such as Norway, Austria, Ireland, France, Denmark, Germany, and Finland participated in PIAAC 2012. The results of the model with item and person characteristics demonstrated that adults tended to be disengaged on very difficult items. Similarly, age has a negative effect on test-taking engagement for adults in several countries such as France and Ireland, while several predictors such as educational attainment, readiness to learn, and the use of ICT skills at home and work had positive effects on test engagement. In addition, females exhibit a higher level of engagement in Norway. Overall, the findings suggested that the effect of the predictors on disengagement depended on the domain and country. So, this study brings further attention that the role of test disengagement should be a prerequisite practice before reaching a conclusion from international large-stake assessments.
{"title":"Silent Predictors of Test Disengagement in PIAAC 2012","authors":"Münevver İLGÜN DİBEK","doi":"10.21031/epod.796626","DOIUrl":"https://doi.org/10.21031/epod.796626","url":null,"abstract":"Although the effects of test disengagement on the validity of the scores obtained from the data set have been examined in many studies, the predictors of the disengaged behaviors received relatively limited scholarly attention in low-stakes assessment, in particular, in international comparison studies. As such, the present study with a twofold purpose sets out to determine the best fitted explanatory item response theory model and examine the predictors of test disengagement. The data were collected by using items measuring literacy and numeracy skills of adults from different countries such as Norway, Austria, Ireland, France, Denmark, Germany, and Finland participated in PIAAC 2012. The results of the model with item and person characteristics demonstrated that adults tended to be disengaged on very difficult items. Similarly, age has a negative effect on test-taking engagement for adults in several countries such as France and Ireland, while several predictors such as educational attainment, readiness to learn, and the use of ICT skills at home and work had positive effects on test engagement. In addition, females exhibit a higher level of engagement in Norway. Overall, the findings suggested that the effect of the predictors on disengagement depended on the domain and country. So, this study brings further attention that the role of test disengagement should be a prerequisite practice before reaching a conclusion from international large-stake assessments.","PeriodicalId":43015,"journal":{"name":"Journal of Measurement and Evaluation in Education and Psychology-EPOD","volume":null,"pages":null},"PeriodicalIF":0.5,"publicationDate":"2020-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"68252975","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Impact of Covariate Variables on Test Equating 协变量对检验等式的影响
IF 0.5 Q4 Social Sciences Pub Date : 2020-11-15 DOI: 10.21031/epod.706835
Çiğdem AKIN ARIKAN
{"title":"Impact of Covariate Variables on Test Equating","authors":"Çiğdem AKIN ARIKAN","doi":"10.21031/epod.706835","DOIUrl":"https://doi.org/10.21031/epod.706835","url":null,"abstract":"","PeriodicalId":43015,"journal":{"name":"Journal of Measurement and Evaluation in Education and Psychology-EPOD","volume":null,"pages":null},"PeriodicalIF":0.5,"publicationDate":"2020-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"68253099","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Investigation of the Effect of Missing Data Handling Methods on Measurement Invariance of Multi-Dimensional Structures 缺失数据处理方法对多维结构测量不变性影响的研究
IF 0.5 Q4 Social Sciences Pub Date : 2020-09-21 DOI: 10.21031/EPOD.749370
Mehmet Ali Işıkoğlu, B. Atar
The purpose of this study was to compare the missing data handling methods on measurement invariance of multi-dimensional structures. For this purpose, data of 10857 students who participated in PISA 2015 administration from Turkey and Singapore and fully responded to the items related to affective characteristics of science literacy was used. Data with different percentages of missing data (5%, 10%, and 20% missing data) were generated from the complete data set with missing completely at random (MCAR) mechanism. In all data sets, missing data was completed with listwise deletion (LD), serial mean imputation (SMI), regression imputation (RI), expectation maximization (EM), and multiple imputation (MI) methods. Measurement invariance of the construct being measured between countries on completed data sets was investigated with multiple-group confirmatory factor analysis (MG-CFA). Findings from each dataset were compared with reference values. In the results of the study, RI and MI methods in the data set with 5% missing, EM method in the data set with 10% missing, and MI method in the data set with 20% missing gave the more similar results to the reference values than the other methods.
本研究的目的是比较多维结构测量不变性的缺失数据处理方法。为此,使用了来自土耳其和新加坡的10857名参与2015年PISA管理的学生的数据,他们对与科学素养情感特征相关的项目做出了充分的反应。具有不同百分比缺失数据(5%、10%和20%缺失数据)的数据是从具有完全随机缺失(MCAR)机制的完整数据集生成的。在所有数据集中,缺失数据采用列表删除(LD)、序列平均插补(SMI)、回归插补(RI)、期望最大化(EM)和多重插补(MI)方法完成。使用多组验证性因子分析(MG-CFA)研究了在已完成的数据集上国家之间测量的结构的测量不变性。将每个数据集的结果与参考值进行比较。在研究结果中,与其他方法相比,缺失5%数据集中的RI和MI方法、缺失10%数据集中的EM方法和缺失20%数据集中的MI方法给出了与参考值更相似的结果。
{"title":"Investigation of the Effect of Missing Data Handling Methods on Measurement Invariance of Multi-Dimensional Structures","authors":"Mehmet Ali Işıkoğlu, B. Atar","doi":"10.21031/EPOD.749370","DOIUrl":"https://doi.org/10.21031/EPOD.749370","url":null,"abstract":"The purpose of this study was to compare the missing data handling methods on measurement invariance of multi-dimensional structures. For this purpose, data of 10857 students who participated in PISA 2015 administration from Turkey and Singapore and fully responded to the items related to affective characteristics of science literacy was used. Data with different percentages of missing data (5%, 10%, and 20% missing data) were generated from the complete data set with missing completely at random (MCAR) mechanism. In all data sets, missing data was completed with listwise deletion (LD), serial mean imputation (SMI), regression imputation (RI), expectation maximization (EM), and multiple imputation (MI) methods. Measurement invariance of the construct being measured between countries on completed data sets was investigated with multiple-group confirmatory factor analysis (MG-CFA). Findings from each dataset were compared with reference values. In the results of the study, RI and MI methods in the data set with 5% missing, EM method in the data set with 10% missing, and MI method in the data set with 20% missing gave the more similar results to the reference values than the other methods.","PeriodicalId":43015,"journal":{"name":"Journal of Measurement and Evaluation in Education and Psychology-EPOD","volume":null,"pages":null},"PeriodicalIF":0.5,"publicationDate":"2020-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43510060","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Journal of Measurement and Evaluation in Education and Psychology-EPOD
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1