首页 > 最新文献

Educational Research and Evaluation最新文献

英文 中文
Is the relationship between socioeconomic status (SES) and student achievement causal? Considering student and parent abilities 社会经济地位(SES)和学生成绩之间的关系是因果关系吗?考虑学生和家长的能力
IF 1.4 Q1 EDUCATION & EDUCATIONAL RESEARCH Pub Date : 2020-11-16 DOI: 10.1080/13803611.2021.1968442
G. Marks
ABSTRACT Most studies on the relationship between students’ socioeconomic status (SES) and student achievement assume that its effects are sizable and causal. A large variety of theoretical explanations have been proposed. However, the SES–achievement association may reflect, to some extent, the inter-relationships of parents’ abilities, SES, children’s abilities, and student achievement. The purpose of this study is to quantify the role of SES vis-à-vis child and parents’ abilities, and prior achievement. Analyses of a covariance matrix that includes supplementary correlations for fathers and mothers’ abilities derived from the literature indicate that more than half of the SES–achievement association can be accounted for by parents’ abilities. SES coefficients decline further with the addition of child’s abilities. With the addition of prior achievement, the SES coefficients are trivial implying that SES has little or no contemporaneous effects. These findings are not compatible with standard theoretical explanations for SES inequalities in achievement.
大多数关于学生社会经济地位(SES)与学生成绩关系的研究都假设其影响是相当大的和因果的。人们提出了各种各样的理论解释。然而,社会经济地位与成绩的关联可能在一定程度上反映了父母能力、社会经济地位、孩子能力和学生成绩之间的相互关系。本研究的目的是量化社会经济地位对-à-vis儿童和家长的能力,以及先前成就的作用。对协方差矩阵的分析,包括来自文献的父亲和母亲能力的补充相关性,表明超过一半的ses -成就关联可以由父母的能力来解释。随着儿童能力的增加,经济社会地位系数进一步下降。加上先前的成就,社会经济地位系数是微不足道的,这意味着社会经济地位很少或没有同期影响。这些发现与社会经济地位在成就上的不平等的标准理论解释不相容。
{"title":"Is the relationship between socioeconomic status (SES) and student achievement causal? Considering student and parent abilities","authors":"G. Marks","doi":"10.1080/13803611.2021.1968442","DOIUrl":"https://doi.org/10.1080/13803611.2021.1968442","url":null,"abstract":"ABSTRACT Most studies on the relationship between students’ socioeconomic status (SES) and student achievement assume that its effects are sizable and causal. A large variety of theoretical explanations have been proposed. However, the SES–achievement association may reflect, to some extent, the inter-relationships of parents’ abilities, SES, children’s abilities, and student achievement. The purpose of this study is to quantify the role of SES vis-à-vis child and parents’ abilities, and prior achievement. Analyses of a covariance matrix that includes supplementary correlations for fathers and mothers’ abilities derived from the literature indicate that more than half of the SES–achievement association can be accounted for by parents’ abilities. SES coefficients decline further with the addition of child’s abilities. With the addition of prior achievement, the SES coefficients are trivial implying that SES has little or no contemporaneous effects. These findings are not compatible with standard theoretical explanations for SES inequalities in achievement.","PeriodicalId":47025,"journal":{"name":"Educational Research and Evaluation","volume":"26 1","pages":"344 - 367"},"PeriodicalIF":1.4,"publicationDate":"2020-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44764218","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Methods, understandings, and expressions of causality in educational research 教育研究中因果关系的方法、理解和表达
IF 1.4 Q1 EDUCATION & EDUCATIONAL RESEARCH Pub Date : 2020-11-16 DOI: 10.1080/13803611.2021.1991643
K. Morrison, G. P. van der Werf
The opening pages of Pearl and Mackenzie’s volume The Book of Why: The New Science of Cause and Effect (2018) herald their captivating romp through causality by referring to a “ladder of causation” (p. 28) that starts from association (by seeing and observing), moves up to intervention (by doing and intervening), and thence to counterfactuals (by imagining, retrospection, and understanding). Each rung of the ladder establishes causality more certainly. Humans think causally. Causality can be studied by many methods. Here, Pearl and Mackenzie (2018) state that statistical analysis does not simply concern data and their methods of analysis; rather, there is a need for an “understanding of the process that produces the data” (p. 85). Such “understanding” comes from introducing causality, as causality yields something additional to the original data. “Methods” of data analysis are informed by an “understanding” of causality, as this Editorial shows. Pearl and Mackenzie write that if we remove the understanding of causation from statistical analysis, all that we are left with is data reduction, which does not tell us much. The papers in this issue move forward from “methods” to “understanding” data with regard to causality. Further, the Editorial indicates how easily it is to find expressions of causality in articles; this should caution researchers to take care in the wording that they use. The Editorial below draws attention to wording in deliberately italicising causal words in quoting from the articles in this issue. For example, is causality really being demonstrated, or, like Pearl and MacKenzie’s lowest rung of the ladder, is there merely association? Causality, be it post hoc or ante hoc, is self-evidently important in education. However, how we adduce causality is far from straightforward, and the papers in this issue yield insights into, and cautions concerning, claims for, and demonstrations of, causality. The papers here indicate methods, challenges, outcomes, and benefits of studying causality. The challenges of “methods” and “understanding” when investigating causality are legion. Witness, for example, in the perennial search for causality, its differences from association, prediction, explanation, inference, influence, correlation, accounting for, correspondence to, purposiveness, and a whole armoury of other words. Look at the dangers of working with mediating, confounding, and moderating variables, transitivity, or controlling out almost everything such that what remains is very little. Wrestle with underdetermination, overdetermination, supervenience, and the difficulties of mereology. Consider the challenges of probabilistic causation and Bayesian approaches, leavened by multilevel causal modelling. Add to these the context-dense, variable-rich, causally complex world of education, and the attraction of Pearl and MacKenzie’s (2018) “childlike simplicity” (p. 39) of a causal diagram evaporates in front of our eyes. Little wonder it i
珀尔和麦肯齐的《为什么之书:因果关系的新科学》(2018)一书的开头几页通过提到“因果关系的阶梯”(第28页),预示着他们在因果关系中的迷人嬉戏,“因果关系阶梯”从联想(通过看和观察)开始,向上移动到干预(通过做和干预),然后是反事实(通过想象、回顾和理解)。阶梯的每一级都更确定地建立了因果关系。人类的思考是因果的。因果关系可以通过多种方法来研究。在这里,Pearl和Mackenzie(2018)指出,统计分析不仅仅涉及数据及其分析方法;相反,需要“理解产生数据的过程”(第85页)。这种“理解”来自于引入因果关系,因为因果关系在原始数据的基础上产生了额外的东西。正如这篇社论所示,数据分析的“方法”是由对因果关系的“理解”决定的。珀尔和麦肯齐写道,如果我们从统计分析中去除对因果关系的理解,我们只剩下数据减少,这并不能告诉我们什么。本期的论文从“方法”转向了“理解”关于因果关系的数据。此外,社论指出,在文章中找到因果关系的表达是多么容易;这应该提醒研究人员注意他们使用的措辞。下面的社论提请注意在引用本期文章时故意用斜体表示因果词的措辞。例如,因果关系真的得到了证明吗?或者,就像珀尔和麦肯齐的最底层一样,仅仅存在关联吗?因果关系,无论是事后的还是事前的,在教育中都是不言而喻的重要。然而,我们如何援引因果关系远非易事,本期的论文对因果关系的主张和证明提出了见解和警告。本文指出了研究因果关系的方法、挑战、结果和好处。在调查因果关系时,“方法”和“理解”的挑战是巨大的。例如,在长期寻找因果关系的过程中,见证了因果关系与联想、预测、解释、推理、影响、相关性、解释、对应性、目的性以及其他词汇的差异。看看使用中介、混淆和调节变量、传递性或控制几乎所有东西的危险,这样剩下的就很少了。与不确定、过度确定、偶然性和语言学的困难作斗争。考虑概率因果关系和贝叶斯方法的挑战,这些方法受到多层次因果模型的影响。再加上上下文密集、变量丰富、因果关系复杂的教育世界,珀尔和麦肯齐(2018)的因果图“童真”(第39页)的吸引力在我们眼前消失了。因此,难怪在教育领域研究因果关系的作者措辞非常谨慎。这有助于他们避免承担或
{"title":"Methods, understandings, and expressions of causality in educational research","authors":"K. Morrison, G. P. van der Werf","doi":"10.1080/13803611.2021.1991643","DOIUrl":"https://doi.org/10.1080/13803611.2021.1991643","url":null,"abstract":"The opening pages of Pearl and Mackenzie’s volume The Book of Why: The New Science of Cause and Effect (2018) herald their captivating romp through causality by referring to a “ladder of causation” (p. 28) that starts from association (by seeing and observing), moves up to intervention (by doing and intervening), and thence to counterfactuals (by imagining, retrospection, and understanding). Each rung of the ladder establishes causality more certainly. Humans think causally. Causality can be studied by many methods. Here, Pearl and Mackenzie (2018) state that statistical analysis does not simply concern data and their methods of analysis; rather, there is a need for an “understanding of the process that produces the data” (p. 85). Such “understanding” comes from introducing causality, as causality yields something additional to the original data. “Methods” of data analysis are informed by an “understanding” of causality, as this Editorial shows. Pearl and Mackenzie write that if we remove the understanding of causation from statistical analysis, all that we are left with is data reduction, which does not tell us much. The papers in this issue move forward from “methods” to “understanding” data with regard to causality. Further, the Editorial indicates how easily it is to find expressions of causality in articles; this should caution researchers to take care in the wording that they use. The Editorial below draws attention to wording in deliberately italicising causal words in quoting from the articles in this issue. For example, is causality really being demonstrated, or, like Pearl and MacKenzie’s lowest rung of the ladder, is there merely association? Causality, be it post hoc or ante hoc, is self-evidently important in education. However, how we adduce causality is far from straightforward, and the papers in this issue yield insights into, and cautions concerning, claims for, and demonstrations of, causality. The papers here indicate methods, challenges, outcomes, and benefits of studying causality. The challenges of “methods” and “understanding” when investigating causality are legion. Witness, for example, in the perennial search for causality, its differences from association, prediction, explanation, inference, influence, correlation, accounting for, correspondence to, purposiveness, and a whole armoury of other words. Look at the dangers of working with mediating, confounding, and moderating variables, transitivity, or controlling out almost everything such that what remains is very little. Wrestle with underdetermination, overdetermination, supervenience, and the difficulties of mereology. Consider the challenges of probabilistic causation and Bayesian approaches, leavened by multilevel causal modelling. Add to these the context-dense, variable-rich, causally complex world of education, and the attraction of Pearl and MacKenzie’s (2018) “childlike simplicity” (p. 39) of a causal diagram evaporates in front of our eyes. Little wonder it i","PeriodicalId":47025,"journal":{"name":"Educational Research and Evaluation","volume":"26 1","pages":"339 - 343"},"PeriodicalIF":1.4,"publicationDate":"2020-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48385374","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Understanding and using challenging educational theories 理解和使用具有挑战性的教育理论
IF 1.4 Q1 EDUCATION & EDUCATIONAL RESEARCH Pub Date : 2020-11-16 DOI: 10.1080/13803611.2021.1969810
Y. Tan
{"title":"Understanding and using challenging educational theories","authors":"Y. Tan","doi":"10.1080/13803611.2021.1969810","DOIUrl":"https://doi.org/10.1080/13803611.2021.1969810","url":null,"abstract":"","PeriodicalId":47025,"journal":{"name":"Educational Research and Evaluation","volume":"26 1","pages":"460 - 462"},"PeriodicalIF":1.4,"publicationDate":"2020-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45407763","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 9
Does test-taking motivation predict test results in a high-stakes testing context? 在高风险的测试环境中,参加测试的动机是否能预测测试结果?
IF 1.4 Q1 EDUCATION & EDUCATIONAL RESEARCH Pub Date : 2020-11-16 DOI: 10.1080/13803611.2021.1949355
G. Silm, O. Must, Karin Täht, M. Pedaste
ABSTRACT Test-taking motivation (TTM) has been associated with test performance in low-stakes testing contexts. However, there have been few studies about TTM in high-stakes testing contexts, and these have contradictory results. Our aim was to explore the relationship between test-taking effort and test performance in a real-life high-stakes testing context (n = 1,515). We collected time-based and self-reported data about test-taking effort and used a structural equation model (SEM) to predict test performance. We found that the motivational indicators added about 15% of predictive power to the SEM model, where gender and previous performance had been controlled for. Altogether, the SEM model predicted 69% of the variance in test results. We compared the findings to previous studies and concluded that the possible effect of TTM should be considered in various testing contexts, whether low-stakes or high-stakes.
在低风险测试环境中,应试动机(TTM)与测试表现有关。然而,关于TTM在高风险测试环境中的研究很少,而且这些研究的结果相互矛盾。我们的目的是探索在现实生活中的高风险考试环境中,考试努力和考试成绩之间的关系(n = 1515)。我们收集了基于时间和自我报告的考试努力数据,并使用结构方程模型(SEM)来预测考试成绩。我们发现动机指标为SEM模型增加了约15%的预测能力,其中性别和以前的表现已被控制。总的来说,SEM模型预测了测试结果中69%的方差。我们将研究结果与之前的研究进行了比较,并得出结论,TTM的可能影响应该在各种测试环境中考虑,无论是低风险还是高风险。
{"title":"Does test-taking motivation predict test results in a high-stakes testing context?","authors":"G. Silm, O. Must, Karin Täht, M. Pedaste","doi":"10.1080/13803611.2021.1949355","DOIUrl":"https://doi.org/10.1080/13803611.2021.1949355","url":null,"abstract":"ABSTRACT Test-taking motivation (TTM) has been associated with test performance in low-stakes testing contexts. However, there have been few studies about TTM in high-stakes testing contexts, and these have contradictory results. Our aim was to explore the relationship between test-taking effort and test performance in a real-life high-stakes testing context (n = 1,515). We collected time-based and self-reported data about test-taking effort and used a structural equation model (SEM) to predict test performance. We found that the motivational indicators added about 15% of predictive power to the SEM model, where gender and previous performance had been controlled for. Altogether, the SEM model predicted 69% of the variance in test results. We compared the findings to previous studies and concluded that the possible effect of TTM should be considered in various testing contexts, whether low-stakes or high-stakes.","PeriodicalId":47025,"journal":{"name":"Educational Research and Evaluation","volume":"26 1","pages":"387 - 413"},"PeriodicalIF":1.4,"publicationDate":"2020-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/13803611.2021.1949355","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45569696","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
The impact of students’ test-taking effort on growth estimates in low-stakes educational assessments 在低风险教育评估中,学生的考试努力对增长估计的影响
IF 1.4 Q1 EDUCATION & EDUCATIONAL RESEARCH Pub Date : 2020-11-16 DOI: 10.1080/13803611.2021.1977152
S. Yildirim-Erbasli, O. Bulut
ABSTRACT This study investigated the impact of students’ test-taking effort on their growth estimates in reading. The sample consisted of 7,602 students (Grades 1 to 4) in the United States who participated in the fall and spring administrations of a computer-based reading assessment. First, a new response dataset was created by flagging both rapid-guessing and slow-responding behaviours and recoding these non-effortful responses as missing. Second, students’ academic growth (i.e., daily increase in ability levels) from fall to spring was calculated based on their original responses and responses in the new dataset excluding non-effortful responses. The results indicated that students’ growth estimates changed significantly after recoding non-effortful responses as missing. Also, the difference in the growth estimates varied depending on the grade level. Overall, students’ test-taking effort appeared to be influential in the estimation of students’ reading growth. Implications for practice were discussed.
摘要本研究探讨了学生应试努力对其阅读能力增长预估的影响。样本包括7602名美国学生(一年级到四年级),他们参加了秋季和春季的计算机阅读评估管理。首先,通过标记快速猜测和慢反应行为,并将这些不费力的反应重新编码为缺失,创建了一个新的反应数据集。其次,学生从秋季到春季的学业增长(即能力水平的每日增长)是基于他们的原始回答和新数据集中的回答计算的,不包括不努力的回答。结果表明,在将不费力的回答重新编码为缺失后,学生的成长估计发生了显着变化。此外,增长估计的差异因年级水平而异。总体而言,学生的应试努力似乎对学生阅读增长的估计有影响。讨论了对实践的启示。
{"title":"The impact of students’ test-taking effort on growth estimates in low-stakes educational assessments","authors":"S. Yildirim-Erbasli, O. Bulut","doi":"10.1080/13803611.2021.1977152","DOIUrl":"https://doi.org/10.1080/13803611.2021.1977152","url":null,"abstract":"ABSTRACT This study investigated the impact of students’ test-taking effort on their growth estimates in reading. The sample consisted of 7,602 students (Grades 1 to 4) in the United States who participated in the fall and spring administrations of a computer-based reading assessment. First, a new response dataset was created by flagging both rapid-guessing and slow-responding behaviours and recoding these non-effortful responses as missing. Second, students’ academic growth (i.e., daily increase in ability levels) from fall to spring was calculated based on their original responses and responses in the new dataset excluding non-effortful responses. The results indicated that students’ growth estimates changed significantly after recoding non-effortful responses as missing. Also, the difference in the growth estimates varied depending on the grade level. Overall, students’ test-taking effort appeared to be influential in the estimation of students’ reading growth. Implications for practice were discussed.","PeriodicalId":47025,"journal":{"name":"Educational Research and Evaluation","volume":"26 1","pages":"368 - 386"},"PeriodicalIF":1.4,"publicationDate":"2020-11-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43302422","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
Differential rapid responding across language and cultural groups 不同语言和文化群体的快速反应差异
IF 1.4 Q1 EDUCATION & EDUCATIONAL RESEARCH Pub Date : 2020-08-17 DOI: 10.1080/13803611.2021.1963941
Hongwen Guo, Kadriye Ercikan
ABSTRACT Rapid response behaviour, a type of test disengagement, cannot be interpreted as a true indicator of the targeted constructs and may compromise score accuracy as well as score validity for interpretation. Rapid responding may be due to multiple factors for diverse populations. In this study, using Programme for International Student Assessment (PISA) 2018 Science data, we examined the comparability of rapid response behaviours for nine different language and cultural groups. Statistical methods were developed to flag rapid responses on different item types and to test differential rapid responding rates between groups. Results showed that rapid response rates and their association with performance were different among the studied groups. However, regardless of students’ group membership, a rapid response led to a chance correct rate, and it had a negative impact on performance. The results also indicated limited impact of different test engagement on performance ranking for the studied groups.
快速反应行为是一种测试脱离,不能被解释为目标构念的真实指标,可能会损害分数的准确性和解释的分数效度。快速反应可能是由于不同人群的多种因素造成的。在这项研究中,我们使用2018年国际学生评估项目(PISA)的科学数据,研究了9个不同语言和文化群体的快速反应行为的可比性。我们开发了统计方法来标记对不同项目类型的快速反应,并测试组间不同的快速反应率。结果表明,快速反应率及其与成绩的关系在研究小组中是不同的。然而,无论学生是哪个小组成员,快速反应都会导致机会正确率,并对成绩产生负面影响。研究结果还表明,不同的测试参与度对研究小组的表现排名影响有限。
{"title":"Differential rapid responding across language and cultural groups","authors":"Hongwen Guo, Kadriye Ercikan","doi":"10.1080/13803611.2021.1963941","DOIUrl":"https://doi.org/10.1080/13803611.2021.1963941","url":null,"abstract":"ABSTRACT Rapid response behaviour, a type of test disengagement, cannot be interpreted as a true indicator of the targeted constructs and may compromise score accuracy as well as score validity for interpretation. Rapid responding may be due to multiple factors for diverse populations. In this study, using Programme for International Student Assessment (PISA) 2018 Science data, we examined the comparability of rapid response behaviours for nine different language and cultural groups. Statistical methods were developed to flag rapid responses on different item types and to test differential rapid responding rates between groups. Results showed that rapid response rates and their association with performance were different among the studied groups. However, regardless of students’ group membership, a rapid response led to a chance correct rate, and it had a negative impact on performance. The results also indicated limited impact of different test engagement on performance ranking for the studied groups.","PeriodicalId":47025,"journal":{"name":"Educational Research and Evaluation","volume":"26 1","pages":"302 - 327"},"PeriodicalIF":1.4,"publicationDate":"2020-08-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41439984","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
Six insights regarding test-taking disengagement 关于脱离测试的六点见解
IF 1.4 Q1 EDUCATION & EDUCATIONAL RESEARCH Pub Date : 2020-08-17 DOI: 10.1080/13803611.2021.1963942
S. Wise
ABSTRACT There has been increasing concern about the presence of disengaged test taking in international assessment programmes and its implications for the validity of inferences made regarding a country’s level of educational attainment. This issue has received a growing research interest over the past 20 years, with notable advances in both the measurement of disengagement as well as our understanding of its distortive impact on both individual and aggregated scores. In this paper, I discuss six important insights yielded by this research and their implications for assessment programmes.
摘要人们越来越担心国际评估计划中存在的不参与考试及其对一个国家教育水平推断的有效性的影响。在过去的20年里,这个问题受到了越来越多的研究兴趣,在衡量脱离接触以及我们对其对个人和总分的扭曲影响的理解方面都取得了显著进展。在本文中,我讨论了这项研究得出的六个重要见解及其对评估计划的影响。
{"title":"Six insights regarding test-taking disengagement","authors":"S. Wise","doi":"10.1080/13803611.2021.1963942","DOIUrl":"https://doi.org/10.1080/13803611.2021.1963942","url":null,"abstract":"ABSTRACT There has been increasing concern about the presence of disengaged test taking in international assessment programmes and its implications for the validity of inferences made regarding a country’s level of educational attainment. This issue has received a growing research interest over the past 20 years, with notable advances in both the measurement of disengagement as well as our understanding of its distortive impact on both individual and aggregated scores. In this paper, I discuss six important insights yielded by this research and their implications for assessment programmes.","PeriodicalId":47025,"journal":{"name":"Educational Research and Evaluation","volume":"26 1","pages":"328 - 338"},"PeriodicalIF":1.4,"publicationDate":"2020-08-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46329911","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
Cross-cultural study of test effort in PISA PISA测试努力的跨文化研究
IF 1.4 Q1 EDUCATION & EDUCATIONAL RESEARCH Pub Date : 2020-08-17 DOI: 10.1080/13803611.2021.1963943
Gavin T. L. Brown, Kane Meissel
{"title":"Cross-cultural study of test effort in PISA","authors":"Gavin T. L. Brown, Kane Meissel","doi":"10.1080/13803611.2021.1963943","DOIUrl":"https://doi.org/10.1080/13803611.2021.1963943","url":null,"abstract":"","PeriodicalId":47025,"journal":{"name":"Educational Research and Evaluation","volume":"26 1","pages":"217 - 220"},"PeriodicalIF":1.4,"publicationDate":"2020-08-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44595437","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Within-item response processes as indicators of test-taking effort and motivation 项目内反应过程作为应试努力和动机的指标
IF 1.4 Q1 EDUCATION & EDUCATIONAL RESEARCH Pub Date : 2020-08-17 DOI: 10.1080/13803611.2021.1963940
Erik Lundgren, Hanna Eklöf
ABSTRACT The present study used process data from a computer-based problem-solving task as indications of behavioural level of test-taking effort, and explored how behavioural item-level effort related to overall test performance and self-reported effort. Variables were extracted from raw process data and clustered. Four distinct clusters were obtained and characterised as high effort, medium effort, low effort, and planner. Regression modelling indicated that among students that failed to solve the task, level of effort invested before giving up positively predicted overall test performance. Among students that solved the task, level of effort was instead weakly negatively related to test performance. A low level of behavioural effort before giving up the task was also related to lower self-reported effort. Results suggest that effort invested before giving up provides information about test-takers’ motivation to spend effort on the test. We conclude that process data could augment existing methods of assessing test-taking effort.
本研究使用基于计算机的问题解决任务的过程数据作为测试努力的行为水平的指示,并探讨了行为项目水平的努力如何与整体测试表现和自我报告的努力相关。从原始过程数据中提取变量并进行聚类。得到了四个不同的集群,并以高努力、中等努力、低努力和计划者为特征。回归模型表明,在未能解决任务的学生中,在放弃之前投入的努力程度正预测了整体考试成绩。在完成任务的学生中,努力程度与考试成绩呈弱负相关。在放弃任务之前,低水平的行为努力也与较低的自我报告努力有关。结果表明,在放弃之前投入的努力提供了有关考生在考试中投入努力的动机的信息。我们的结论是,过程数据可以增强现有的评估考试努力的方法。
{"title":"Within-item response processes as indicators of test-taking effort and motivation","authors":"Erik Lundgren, Hanna Eklöf","doi":"10.1080/13803611.2021.1963940","DOIUrl":"https://doi.org/10.1080/13803611.2021.1963940","url":null,"abstract":"ABSTRACT The present study used process data from a computer-based problem-solving task as indications of behavioural level of test-taking effort, and explored how behavioural item-level effort related to overall test performance and self-reported effort. Variables were extracted from raw process data and clustered. Four distinct clusters were obtained and characterised as high effort, medium effort, low effort, and planner. Regression modelling indicated that among students that failed to solve the task, level of effort invested before giving up positively predicted overall test performance. Among students that solved the task, level of effort was instead weakly negatively related to test performance. A low level of behavioural effort before giving up the task was also related to lower self-reported effort. Results suggest that effort invested before giving up provides information about test-takers’ motivation to spend effort on the test. We conclude that process data could augment existing methods of assessing test-taking effort.","PeriodicalId":47025,"journal":{"name":"Educational Research and Evaluation","volume":"26 1","pages":"275 - 301"},"PeriodicalIF":1.4,"publicationDate":"2020-08-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45210217","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 12
How does the number of actions on constructed-response items relate to test-taking effort and performance? 构建的响应项上的操作数量与测试的工作量和性能有何关系?
IF 1.4 Q1 EDUCATION & EDUCATIONAL RESEARCH Pub Date : 2020-08-17 DOI: 10.1080/13803611.2021.1963939
M. Ivanova, M. Michaelides, Hanna Eklöf
ABSTRACT Collecting process data in computer-based assessments provides opportunities to describe examinee behaviour during a test-taking session. The number of actions taken by students while interacting with an item is in this context a variable that has been gaining attention. The present study aims to investigate how the number of actions performed on constructed-response items relates to self-reported effort, performance, and item cluster position in the test. The theory of planned behaviour was used as an interpretative framework. Data from two item clusters of the 2015 Swedish Programme for International Student Assessment (PISA) Science administration were utilised. Results showed that the number of actions was significantly related to performance on the items, self-reported test-taking effort, and cluster position. Latent variable models were examined separately for performance-level groups. Overall, the number of actions performed on constructed-response items as a behavioural indicator in testing situations may be useful in gauging test-taking engagement.
摘要在基于计算机的评估中收集过程数据为描述考生在考试过程中的行为提供了机会。在这种情况下,学生在与项目互动时采取的行动数量是一个越来越受到关注的变量。本研究旨在调查在构建的反应项目上执行的动作数量与自我报告的努力、表现和测试中项目集群位置之间的关系。计划行为理论被用作一个解释框架。使用了2015年瑞典国际学生评估计划(PISA)科学管理的两个项目集群的数据。结果显示,行动次数与项目表现、自我报告的测试努力和集群位置显著相关。对绩效水平组的潜在变量模型进行了单独检查。总体而言,在测试情况下,作为行为指标,对构建的响应项目执行的行动数量可能有助于衡量测试参与度。
{"title":"How does the number of actions on constructed-response items relate to test-taking effort and performance?","authors":"M. Ivanova, M. Michaelides, Hanna Eklöf","doi":"10.1080/13803611.2021.1963939","DOIUrl":"https://doi.org/10.1080/13803611.2021.1963939","url":null,"abstract":"ABSTRACT Collecting process data in computer-based assessments provides opportunities to describe examinee behaviour during a test-taking session. The number of actions taken by students while interacting with an item is in this context a variable that has been gaining attention. The present study aims to investigate how the number of actions performed on constructed-response items relates to self-reported effort, performance, and item cluster position in the test. The theory of planned behaviour was used as an interpretative framework. Data from two item clusters of the 2015 Swedish Programme for International Student Assessment (PISA) Science administration were utilised. Results showed that the number of actions was significantly related to performance on the items, self-reported test-taking effort, and cluster position. Latent variable models were examined separately for performance-level groups. Overall, the number of actions performed on constructed-response items as a behavioural indicator in testing situations may be useful in gauging test-taking engagement.","PeriodicalId":47025,"journal":{"name":"Educational Research and Evaluation","volume":"26 1","pages":"252 - 274"},"PeriodicalIF":1.4,"publicationDate":"2020-08-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46284404","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
期刊
Educational Research and Evaluation
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1