Pub Date : 2021-09-23DOI: 10.1080/0969594X.2021.1929829
John Jerrim
ABSTRACT There is growing concern about the mental wellbeing of young people, including how this is related to national tests. This is a particularly important policy issue in England, where it is claimed that the end of primary Key Stage 2 tests cause schools, pupils and teachers stress. I investigate this issue using data from the Millennium Cohort Study, comparing the wellbeing of pupils in England (measured around the point they are sitting their Key Stage 2 tests) to the rest of the UK (where Key Stage 2 tests are not taken). No evidence is found that the Key Stage 2 tests in England is associated with lower levels of happiness, enjoyment of school, self-esteem or children’s mental wellbeing. Likewise, no evidence is found that children who are happier, more self-confident or with higher levels of wellbeing obtain higher Key Stage 2 test scores.
{"title":"National tests and the wellbeing of primary school pupils: new evidence from the UK","authors":"John Jerrim","doi":"10.1080/0969594X.2021.1929829","DOIUrl":"https://doi.org/10.1080/0969594X.2021.1929829","url":null,"abstract":"ABSTRACT There is growing concern about the mental wellbeing of young people, including how this is related to national tests. This is a particularly important policy issue in England, where it is claimed that the end of primary Key Stage 2 tests cause schools, pupils and teachers stress. I investigate this issue using data from the Millennium Cohort Study, comparing the wellbeing of pupils in England (measured around the point they are sitting their Key Stage 2 tests) to the rest of the UK (where Key Stage 2 tests are not taken). No evidence is found that the Key Stage 2 tests in England is associated with lower levels of happiness, enjoyment of school, self-esteem or children’s mental wellbeing. Likewise, no evidence is found that children who are happier, more self-confident or with higher levels of wellbeing obtain higher Key Stage 2 test scores.","PeriodicalId":51515,"journal":{"name":"Assessment in Education-Principles Policy & Practice","volume":"120 1","pages":"507 - 544"},"PeriodicalIF":3.2,"publicationDate":"2021-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85402705","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-20DOI: 10.1080/0969594X.2021.1976108
Cameron Brooks, Rochelle Burton, F. M. Van der Kleij, A. Carroll, J. Hattie
ABSTRACT This paper reports on the evaluation of the final year of a three-year professional learning intervention underpinned by a student-centred model of feedback. School leaders, teachers, and students in 13 Australian state schools participated in the research. The professional learning was contextualised in Year 3 English, with a focus on writing. Data from school leader, teacher and student interviews, researcher field notes, classroom observations and student assessments were analysed to evaluate the impact of the intervention. Findings show the professional learning intervention was positively valued by teachers and school leaders. Evidence from multiple sources suggests that although there was variability in the nature of implementation, teachers and school leaders overall reported substantial changes to knowledge, skills, and classroom practices. Importantly, the results highlight substantial improvements in student self-regulation and writing achievement, providing evidence of the intervention’s positive impact.
{"title":"Towards student-centred feedback practices: evaluating the impact of a professional learning intervention in primary schools","authors":"Cameron Brooks, Rochelle Burton, F. M. Van der Kleij, A. Carroll, J. Hattie","doi":"10.1080/0969594X.2021.1976108","DOIUrl":"https://doi.org/10.1080/0969594X.2021.1976108","url":null,"abstract":"ABSTRACT This paper reports on the evaluation of the final year of a three-year professional learning intervention underpinned by a student-centred model of feedback. School leaders, teachers, and students in 13 Australian state schools participated in the research. The professional learning was contextualised in Year 3 English, with a focus on writing. Data from school leader, teacher and student interviews, researcher field notes, classroom observations and student assessments were analysed to evaluate the impact of the intervention. Findings show the professional learning intervention was positively valued by teachers and school leaders. Evidence from multiple sources suggests that although there was variability in the nature of implementation, teachers and school leaders overall reported substantial changes to knowledge, skills, and classroom practices. Importantly, the results highlight substantial improvements in student self-regulation and writing achievement, providing evidence of the intervention’s positive impact.","PeriodicalId":51515,"journal":{"name":"Assessment in Education-Principles Policy & Practice","volume":"15 1","pages":"633 - 656"},"PeriodicalIF":3.2,"publicationDate":"2021-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83715967","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-09-17DOI: 10.1080/0969594X.2021.1978387
Xiaopeng Wu, Rongxiu Wu, Yi Zhang, D. Arthur, Hua-Hua Chang
ABSTRACT Learning path and learning progression have received extensive attention from broad disciplines. The existing research In the field of learning path is rarely applied in curriculum learning and teaching. Learning progression is usually constructed through observations, interviews but not quantitative analyses. With 726 Grade 8 students’ mathematical knowledge in TIMSS-2015 as the research object, this research adopted a newly generated assessment theory – cognitive diagnosis assessment as the research tool and exploited methods such as K-means clustering analysis to construct learning path by combing the relationships among the attributes. We obtained the students’ ability θs for each classified group through the 3PL model in the Item Response Theory (IRT) and constructed the learning progressions based on the θs and the attribute relationships. From a data-driven approach, this method has provided a new perspective as well as the data support for the construction of the learning paths and the learning progressions.
{"title":"Research on construction method of learning paths and learning progressions based on cognitive diagnosis assessment","authors":"Xiaopeng Wu, Rongxiu Wu, Yi Zhang, D. Arthur, Hua-Hua Chang","doi":"10.1080/0969594X.2021.1978387","DOIUrl":"https://doi.org/10.1080/0969594X.2021.1978387","url":null,"abstract":"ABSTRACT Learning path and learning progression have received extensive attention from broad disciplines. The existing research In the field of learning path is rarely applied in curriculum learning and teaching. Learning progression is usually constructed through observations, interviews but not quantitative analyses. With 726 Grade 8 students’ mathematical knowledge in TIMSS-2015 as the research object, this research adopted a newly generated assessment theory – cognitive diagnosis assessment as the research tool and exploited methods such as K-means clustering analysis to construct learning path by combing the relationships among the attributes. We obtained the students’ ability θs for each classified group through the 3PL model in the Item Response Theory (IRT) and constructed the learning progressions based on the θs and the attribute relationships. From a data-driven approach, this method has provided a new perspective as well as the data support for the construction of the learning paths and the learning progressions.","PeriodicalId":51515,"journal":{"name":"Assessment in Education-Principles Policy & Practice","volume":"23 1","pages":"657 - 675"},"PeriodicalIF":3.2,"publicationDate":"2021-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82815550","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-08-31DOI: 10.1080/0969594X.2021.1970513
Tony Clark, Heidi Endres
ABSTRACT An effective diagnostic test can play a key role in language learning, allowing strengths and weaknesses in students’ linguistic development to be identified and addressed. This paper describes the online Cambridge English diagnostic test, assessing English grammatical knowledge at A2 level. As most language tests focus on proficiency or achievement, relatively little research into diagnostic assessment has been conducted, and no real agreement exists on what it entails. For learners of approximately 15 years old, the test provides diagnostic feedback on seven grammatical categories at individual and class levels, to improve curriculum and lesson planning and accommodate students’ needs. The test was trialled internationally, using surveys and focus groups. This paper outlines the trial and planned modifications for the next version, in addition to implications for wider practice. It should be clarified that this is a first iteration of the diagnostic test, rather than a fully established final model.
{"title":"Computer-based diagnostic assessment of high school students’ grammar skills with automated feedback – an international trial","authors":"Tony Clark, Heidi Endres","doi":"10.1080/0969594X.2021.1970513","DOIUrl":"https://doi.org/10.1080/0969594X.2021.1970513","url":null,"abstract":"ABSTRACT An effective diagnostic test can play a key role in language learning, allowing strengths and weaknesses in students’ linguistic development to be identified and addressed. This paper describes the online Cambridge English diagnostic test, assessing English grammatical knowledge at A2 level. As most language tests focus on proficiency or achievement, relatively little research into diagnostic assessment has been conducted, and no real agreement exists on what it entails. For learners of approximately 15 years old, the test provides diagnostic feedback on seven grammatical categories at individual and class levels, to improve curriculum and lesson planning and accommodate students’ needs. The test was trialled internationally, using surveys and focus groups. This paper outlines the trial and planned modifications for the next version, in addition to implications for wider practice. It should be clarified that this is a first iteration of the diagnostic test, rather than a fully established final model.","PeriodicalId":51515,"journal":{"name":"Assessment in Education-Principles Policy & Practice","volume":"13 1","pages":"602 - 632"},"PeriodicalIF":3.2,"publicationDate":"2021-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88475544","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-08-17DOI: 10.1080/0969594X.2021.1951164
L. Allal
ABSTRACT This paper describes the enlargement of the initial conception of formative assessment with reference to constructivist, sociocultural and situated theories of learning and the concept of co-regulation. It reviews research on student involvement in formative assessment practices (self-assessment, peer assessment, whole-class discussions of criteria and exemplars) in the area of writing, with a focus on primary school classrooms. Student participation in the co-construction of formative assessment is illustrated by qualitative observations from a study of a writing activity carried out in grades 5 and 6. The observations are discussed in relation to findings from other research and implications are presented for teacher professional development and for future studies of formative assessment of writing.
{"title":"Involving primary school students in the co-construction of formative assessment in support of writing","authors":"L. Allal","doi":"10.1080/0969594X.2021.1951164","DOIUrl":"https://doi.org/10.1080/0969594X.2021.1951164","url":null,"abstract":"ABSTRACT This paper describes the enlargement of the initial conception of formative assessment with reference to constructivist, sociocultural and situated theories of learning and the concept of co-regulation. It reviews research on student involvement in formative assessment practices (self-assessment, peer assessment, whole-class discussions of criteria and exemplars) in the area of writing, with a focus on primary school classrooms. Student participation in the co-construction of formative assessment is illustrated by qualitative observations from a study of a writing activity carried out in grades 5 and 6. The observations are discussed in relation to findings from other research and implications are presented for teacher professional development and for future studies of formative assessment of writing.","PeriodicalId":51515,"journal":{"name":"Assessment in Education-Principles Policy & Practice","volume":"76 1","pages":"584 - 601"},"PeriodicalIF":3.2,"publicationDate":"2021-08-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80528965","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-25DOI: 10.1080/0969594X.2021.1951161
L. Hutchinson, Nancy E. Perry, Jenna D. Shapka
ABSTRACT Self-regulation describes how individuals assess and adapt to demands within and across environments. Research accumulated over the past quarter century identifies self-regulation as a powerful predictor of children’s school success. However, studying young children’s self-regulation in school is challenging. Tools that are easy and efficient to administer, closely linked to curriculum and learning in classrooms, and that do not require self-reports from children are needed. Here we report on the development and validation of the Self-Regulation In School Inventory (SRISI), a teacher-report tool designed to assess typically developing young children’s self-regulation in school. Then, we present data from the SRISI that shows how different targets of self-regulation in school were related to one another, school adjustment, child gender, and achievement. Data were gathered from 28 teachers who provided ratings of 307 kindergarten children’s (age range = 4.96–6.61 years old) self-regulation using the SRISI. An exploratory factor analysis on the SRISI items distinguished three targets of self-regulation in school: ‘Emotion Regulation’, ‘Self-Regulation of/for Learning’ and ‘Socially Responsible Self-Regulation’. Path analysis confirmed the relationship between child gender and ER and SRSR, and between SRL and achievement. Findings are situated within a larger discussion concerning the assessment of young children’s self-regulation in school.
{"title":"Assessing young children’s self-regulation in school contexts","authors":"L. Hutchinson, Nancy E. Perry, Jenna D. Shapka","doi":"10.1080/0969594X.2021.1951161","DOIUrl":"https://doi.org/10.1080/0969594X.2021.1951161","url":null,"abstract":"ABSTRACT Self-regulation describes how individuals assess and adapt to demands within and across environments. Research accumulated over the past quarter century identifies self-regulation as a powerful predictor of children’s school success. However, studying young children’s self-regulation in school is challenging. Tools that are easy and efficient to administer, closely linked to curriculum and learning in classrooms, and that do not require self-reports from children are needed. Here we report on the development and validation of the Self-Regulation In School Inventory (SRISI), a teacher-report tool designed to assess typically developing young children’s self-regulation in school. Then, we present data from the SRISI that shows how different targets of self-regulation in school were related to one another, school adjustment, child gender, and achievement. Data were gathered from 28 teachers who provided ratings of 307 kindergarten children’s (age range = 4.96–6.61 years old) self-regulation using the SRISI. An exploratory factor analysis on the SRISI items distinguished three targets of self-regulation in school: ‘Emotion Regulation’, ‘Self-Regulation of/for Learning’ and ‘Socially Responsible Self-Regulation’. Path analysis confirmed the relationship between child gender and ER and SRSR, and between SRL and achievement. Findings are situated within a larger discussion concerning the assessment of young children’s self-regulation in school.","PeriodicalId":51515,"journal":{"name":"Assessment in Education-Principles Policy & Practice","volume":"75 1","pages":"545 - 583"},"PeriodicalIF":3.2,"publicationDate":"2021-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82205333","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-04DOI: 10.1080/0969594X.2021.1951163
Fumiyo Nakatsuhara, C. Inoue, Vivien Berry, E. Galaczi
ABSTRACT This paper investigates the comparability between the video-conferencing and face-to-face modes of the IELTS Speaking Test in terms of scores and language functions generated by test-takers. Data were collected from 10 trained IELTS examiners and 99 test-takers who took two speaking tests under face-to-face and video-conferencing conditions. Many-facet Rasch Model (MFRM) analysis of test scores indicated that the delivery mode did not make any meaningful difference to test-takers’ scores. An examination of language functions revealed that both modes equally elicited the same language functions except asking for clarification. More test-takers made clarification requests in the video-conferencing mode (63.3%) than in the face-to-face mode (26.7%). Drawing on the findings, as well as practical implications, we extend emerging thinking about video-conferencing speaking assessment and the associated features of this modality in its own right.
{"title":"Video-conferencing speaking tests: do they measure the same construct as face-to-face tests?","authors":"Fumiyo Nakatsuhara, C. Inoue, Vivien Berry, E. Galaczi","doi":"10.1080/0969594X.2021.1951163","DOIUrl":"https://doi.org/10.1080/0969594X.2021.1951163","url":null,"abstract":"ABSTRACT This paper investigates the comparability between the video-conferencing and face-to-face modes of the IELTS Speaking Test in terms of scores and language functions generated by test-takers. Data were collected from 10 trained IELTS examiners and 99 test-takers who took two speaking tests under face-to-face and video-conferencing conditions. Many-facet Rasch Model (MFRM) analysis of test scores indicated that the delivery mode did not make any meaningful difference to test-takers’ scores. An examination of language functions revealed that both modes equally elicited the same language functions except asking for clarification. More test-takers made clarification requests in the video-conferencing mode (63.3%) than in the face-to-face mode (26.7%). Drawing on the findings, as well as practical implications, we extend emerging thinking about video-conferencing speaking assessment and the associated features of this modality in its own right.","PeriodicalId":51515,"journal":{"name":"Assessment in Education-Principles Policy & Practice","volume":"15 1","pages":"369 - 388"},"PeriodicalIF":3.2,"publicationDate":"2021-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90530200","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-04DOI: 10.1080/0969594X.2021.1976107
Lin Zhang, Yan-ping Jin
ABSTRACT The computer-based College English Test─Spoken English Test (CET-SET) in China incorporates interactional competence in its construct by using a non-face-to-face paired discussion task. To build validity arguments for this unique computer-mediated interactive task, the present study investigated the impact of delivery mode on test performance in terms of communication strategy use. Data were collected from 60 test-takers who sat both the computer-based CET-SET and its face-to-face equivalent. Through an analysis of test-taker discourse combined with post-test test-taker interviews, the study examined their use of strategies in the discussion task under each mode. The findings of the study provided empirical evidence for the comparability of the two modes. Some differences, however, emerged in the use of interaction strategies, which were primarily attributed to the presence/absence of the visual elements of oral interaction. Hence, suggestions were made to address the issues concerning the computer-delivered mode.
{"title":"Assessing interactional competence in the computer-based CET-SET: An investigation of the use of communication strategies","authors":"Lin Zhang, Yan-ping Jin","doi":"10.1080/0969594X.2021.1976107","DOIUrl":"https://doi.org/10.1080/0969594X.2021.1976107","url":null,"abstract":"ABSTRACT The computer-based College English Test─Spoken English Test (CET-SET) in China incorporates interactional competence in its construct by using a non-face-to-face paired discussion task. To build validity arguments for this unique computer-mediated interactive task, the present study investigated the impact of delivery mode on test performance in terms of communication strategy use. Data were collected from 60 test-takers who sat both the computer-based CET-SET and its face-to-face equivalent. Through an analysis of test-taker discourse combined with post-test test-taker interviews, the study examined their use of strategies in the discussion task under each mode. The findings of the study provided empirical evidence for the comparability of the two modes. Some differences, however, emerged in the use of interaction strategies, which were primarily attributed to the presence/absence of the visual elements of oral interaction. Hence, suggestions were made to address the issues concerning the computer-delivered mode.","PeriodicalId":51515,"journal":{"name":"Assessment in Education-Principles Policy & Practice","volume":"78 1","pages":"389 - 410"},"PeriodicalIF":3.2,"publicationDate":"2021-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76620762","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-04DOI: 10.1080/0969594X.2021.1951162
Nivja H. de Jong, J. Pacilly, W. Heeren
ABSTRACT Fluency in terms of speed of speech and (lack of) hesitations such as silent and filled pauses (‘uhm’s) is part of oral proficiency. Language assessment rubrics therefore include aspects of fluency. Measuring fluency, however, is highly time-consuming because of the manual labour involved. The current paper aims to automatically measure aspects of L2 fluency, including filled pauses, in both Dutch and English. A revised existing script and a new script for filled pauses are tested on accuracy. We also gauged whether the outcomes of the new script could be used for language assessment purposes by relating the outcomes to human judgements. Without further investigations, the current script should not (yet) be used for the purpose of assessing fluency automatically in (high-stakes) oral proficiency assessment. However, the performance of the scripts for measuring aspects of fluency globally and quickly are promising, especially given their stability in accuracy on new corpora.
{"title":"PRAAT scripts to measure speed fluency and breakdown fluency in speech automatically","authors":"Nivja H. de Jong, J. Pacilly, W. Heeren","doi":"10.1080/0969594X.2021.1951162","DOIUrl":"https://doi.org/10.1080/0969594X.2021.1951162","url":null,"abstract":"ABSTRACT Fluency in terms of speed of speech and (lack of) hesitations such as silent and filled pauses (‘uhm’s) is part of oral proficiency. Language assessment rubrics therefore include aspects of fluency. Measuring fluency, however, is highly time-consuming because of the manual labour involved. The current paper aims to automatically measure aspects of L2 fluency, including filled pauses, in both Dutch and English. A revised existing script and a new script for filled pauses are tested on accuracy. We also gauged whether the outcomes of the new script could be used for language assessment purposes by relating the outcomes to human judgements. Without further investigations, the current script should not (yet) be used for the purpose of assessing fluency automatically in (high-stakes) oral proficiency assessment. However, the performance of the scripts for measuring aspects of fluency globally and quickly are promising, especially given their stability in accuracy on new corpora.","PeriodicalId":51515,"journal":{"name":"Assessment in Education-Principles Policy & Practice","volume":"5 1","pages":"456 - 476"},"PeriodicalIF":3.2,"publicationDate":"2021-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80352472","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2021-07-04DOI: 10.1080/0969594X.2021.1999209
Melissa R. Hunte, Samantha McCormick, Maitree Shah, Clarissa Lau, E. Jang
ABSTRACT Children’s oral language proficiency (OLP) is integral for developing literacy skills. Storytelling or retelling is often used by parents and educators to elicit children’s OLP, yet it is less commonly used for assessment purposes. Leveraged by natural language processing and machine learning, this study examined the extent to which computational linguistic and acoustic indices predict human ratings of children’s (n=184 aged 9 to 11) OLP using two story retell stimuli presented in written and aural forms. Human raters scored children’s OLP on five oral proficiency criteria: vocabulary, grammar, idea development, task-fulfilment, and speech delivery, using a 4-point scale, and linguistic and acoustic features were used to predict each criterion. Results showed the efficacy of automated indices to predict human scores of children’s OLP. This study calls for attention to discrepancies in human and machine speech delivery scores and stimulus effects on story retelling performance among children of different language backgrounds.
{"title":"Investigating the potential of NLP-driven linguistic and acoustic features for predicting human scores of children’s oral language proficiency","authors":"Melissa R. Hunte, Samantha McCormick, Maitree Shah, Clarissa Lau, E. Jang","doi":"10.1080/0969594X.2021.1999209","DOIUrl":"https://doi.org/10.1080/0969594X.2021.1999209","url":null,"abstract":"ABSTRACT Children’s oral language proficiency (OLP) is integral for developing literacy skills. Storytelling or retelling is often used by parents and educators to elicit children’s OLP, yet it is less commonly used for assessment purposes. Leveraged by natural language processing and machine learning, this study examined the extent to which computational linguistic and acoustic indices predict human ratings of children’s (n=184 aged 9 to 11) OLP using two story retell stimuli presented in written and aural forms. Human raters scored children’s OLP on five oral proficiency criteria: vocabulary, grammar, idea development, task-fulfilment, and speech delivery, using a 4-point scale, and linguistic and acoustic features were used to predict each criterion. Results showed the efficacy of automated indices to predict human scores of children’s OLP. This study calls for attention to discrepancies in human and machine speech delivery scores and stimulus effects on story retelling performance among children of different language backgrounds.","PeriodicalId":51515,"journal":{"name":"Assessment in Education-Principles Policy & Practice","volume":"16 1","pages":"477 - 505"},"PeriodicalIF":3.2,"publicationDate":"2021-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90237094","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：