首页 > 最新文献

Studies in Language Assessment最新文献

英文 中文
Investigating cognitive processes in different item formats in reading tests through eye-tracking and verbal protocols 通过眼动追踪和语言协议研究阅读测试中不同项目格式的认知过程
Q4 LINGUISTICS Pub Date : 2022-01-01 DOI: 10.58379/uixk2220
Hatice Akgün, A. Ünaldı
This study has investigated the differences in cognitive processes that test-takers undergo while answering reading comprehension questions in multiple-choice and open-ended short answer formats. For this purpose, data were collected from a group of undergraduate students in an English medium university through eye-tracking technology, immediate retrospective verbal protocols, and short semi-structured interviews. The results showed that the participants used careful reading skills more and comprehended the text more thoroughly in the open-ended format. However, in the MC format, they read less carefully and used more test-taking strategies. These findings contribute to the ongoing discussion on how item format can alter the cognitive processes in a reading comprehension test and confirm the effectiveness of eye-tracking in unveiling cognitive processes in combination with qualitative methods. This study has implications for reading test development.
本研究考察了在选择题和开放式简答题中,考生在回答阅读理解题时认知过程的差异。为此,本研究通过眼动追踪技术、即时回顾性口头协议和简短的半结构化访谈,从一所英语媒体大学的一组本科生中收集数据。结果表明,参与者更多地使用了仔细阅读技巧,对开放式文本的理解也更彻底。然而,在MC模式下,他们阅读时不那么仔细,使用了更多的应试策略。这些发现有助于对阅读理解测试中项目格式如何改变认知过程的讨论,并证实眼动追踪与定性方法相结合在揭示认知过程方面的有效性。本研究对阅读测试的发展具有启示意义。
{"title":"Investigating cognitive processes in different item formats in reading tests through eye-tracking and verbal protocols","authors":"Hatice Akgün, A. Ünaldı","doi":"10.58379/uixk2220","DOIUrl":"https://doi.org/10.58379/uixk2220","url":null,"abstract":"This study has investigated the differences in cognitive processes that test-takers undergo while answering reading comprehension questions in multiple-choice and open-ended short answer formats. For this purpose, data were collected from a group of undergraduate students in an English medium university through eye-tracking technology, immediate retrospective verbal protocols, and short semi-structured interviews. The results showed that the participants used careful reading skills more and comprehended the text more thoroughly in the open-ended format. However, in the MC format, they read less carefully and used more test-taking strategies. These findings contribute to the ongoing discussion on how item format can alter the cognitive processes in a reading comprehension test and confirm the effectiveness of eye-tracking in unveiling cognitive processes in combination with qualitative methods. This study has implications for reading test development.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":"22 5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87113410","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Exploring shared and individual assessment of paired oral interactions 探讨配对口头互动的共同和个人评估
Q4 LINGUISTICS Pub Date : 2022-01-01 DOI: 10.58379/homq5772
Pakize Uludag, Kim McDonough, P. Trofimovich
Studies concerning the assessment of second language (L2) paired oral interaction to date have investigated interactional patterns that emerge from paired oral tests and identified the factors that create variability in an individual’s test scores. However, less work has addressed concerns about whether scores should be shared or individual when assessing L2 speakers’ oral interactions. Therefore, the present study compared shared and individual assessment of L2 English paired oral task performances. Paired oral interaction episodes were sampled from a larger corpus of university-level L2 speakers engaged in paired speaking tasks and were assessed by 60 raters who were randomly assigned to rate Speaker A, Speaker B, or both speakers. To avoid any possible rating effects due to rating stimuli, half the raters evaluated audio recordings while the other half assessed video recordings. The raters used an analytic rubric with four domains: discourse management, collaborative communication, content development, and language accuracy and complexity. Comparison of raters’ scores revealed that individual discourse management ratings were significantly higher than shared ratings for both members of the pair regardless of the rating modality (audio vs. video). Implications for assessing pair interactions are discussed.
迄今为止,有关第二语言(L2)配对口语互动评估的研究已经调查了配对口语测试中出现的互动模式,并确定了造成个体测试成绩差异的因素。然而,在评估二语说话者的口语互动时,分数是应该共享还是应该单独计算,这方面的研究较少。因此,本研究比较了二语英语配对口语任务表现的共同评估和个人评估。配对的口语互动片段是从参与配对口语任务的大学水平的第二语言说话者的较大语料库中抽取的,并由60名评分者进行评估,这些评分者被随机分配给说话者a、说话者B或两个说话者。为了避免评分刺激造成的任何可能的评分影响,一半的评分者评估录音,另一半评估录像。评分者使用了一个包含四个领域的分析标准:话语管理、协作交流、内容开发、语言准确性和复杂性。评分者得分的比较显示,无论评分方式(音频还是视频)如何,两人的个人话语管理评分明显高于共同评分。讨论了评估对相互作用的含义。
{"title":"Exploring shared and individual assessment of paired oral interactions","authors":"Pakize Uludag, Kim McDonough, P. Trofimovich","doi":"10.58379/homq5772","DOIUrl":"https://doi.org/10.58379/homq5772","url":null,"abstract":"Studies concerning the assessment of second language (L2) paired oral interaction to date have investigated interactional patterns that emerge from paired oral tests and identified the factors that create variability in an individual’s test scores. However, less work has addressed concerns about whether scores should be shared or individual when assessing L2 speakers’ oral interactions. Therefore, the present study compared shared and individual assessment of L2 English paired oral task performances. Paired oral interaction episodes were sampled from a larger corpus of university-level L2 speakers engaged in paired speaking tasks and were assessed by 60 raters who were randomly assigned to rate Speaker A, Speaker B, or both speakers. To avoid any possible rating effects due to rating stimuli, half the raters evaluated audio recordings while the other half assessed video recordings. The raters used an analytic rubric with four domains: discourse management, collaborative communication, content development, and language accuracy and complexity. Comparison of raters’ scores revealed that individual discourse management ratings were significantly higher than shared ratings for both members of the pair regardless of the rating modality (audio vs. video). Implications for assessing pair interactions are discussed.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":"70 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81053547","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
College English Test–Spoken English Test (CET-SET) 大学英语考试(CET-SET)
Q4 LINGUISTICS Pub Date : 2022-01-01 DOI: 10.58379/oofm7473
Lin Zhang
n/a
N/A
{"title":"College English Test–Spoken English Test (CET-SET)","authors":"Lin Zhang","doi":"10.58379/oofm7473","DOIUrl":"https://doi.org/10.58379/oofm7473","url":null,"abstract":"<jats:p>n/a</jats:p>","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":"17 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82694277","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 10
The meaningfulness of two curriculum-based national tests of English as a foreign language 两个以课程为基础的国家外语英语考试的意义
Q4 LINGUISTICS Pub Date : 2022-01-01 DOI: 10.58379/dqvs8821
R. Hildén, Johanna Pulkkinen, Juhani Rautopuro
This paper addresses the aspect of the meaningfulness of a national assessment in English as a foreign language, applying the fairness framework proposed by Kunnan (2018). We compared students’ performance on two receptive language skills of listening and reading on two subsequent national evaluations in Finland, taken by students at the end of compulsory basic education and at the end of general upper secondary education, respectively. The research questions focus on (1) the relationship between students’ performance on the two tests and their gender, language of schooling and parents’ educational level, and (2) the relationship between the students’ receptive language proficiency at the end of basic education and general upper secondary education. The data were analysed using linear regression and quantile regression analyses. The effect of background variables on the proficiency was stronger for low-performing students than for high-performing students. Moreover, students’ proficiency on the receptive skills at the end of basic education well predicted that at the end of general upper secondary education across several points in the score distribution. The findings also indicated persistent challenges with respect to educational equality and equity that requires ongoing attention by policy-makers and test designers.
本文采用Kunnan(2018)提出的公平性框架,探讨了英语作为外语的国家评估的意义。我们比较了芬兰学生在随后的两次国家评估中听力和阅读两种接受性语言技能的表现,分别由义务基础教育结束时和普通高中教育结束时的学生进行。研究问题主要集中在:(1)学生在两项测试中的表现与性别、学校语言和家长教育水平的关系;(2)学生在基础教育结束时的接受性语言能力与普通高中教育之间的关系。采用线性回归和分位数回归分析对数据进行分析。背景变量对低水平学生的熟练程度的影响大于高水平学生。此外,学生在基础教育结束时的接受技能熟练程度很好地预测了在普通高中教育结束时得分分布的几个点。调查结果还表明,在教育平等和公平方面存在持续挑战,需要政策制定者和考试设计者持续关注。
{"title":"The meaningfulness of two curriculum-based national tests of English as a foreign language","authors":"R. Hildén, Johanna Pulkkinen, Juhani Rautopuro","doi":"10.58379/dqvs8821","DOIUrl":"https://doi.org/10.58379/dqvs8821","url":null,"abstract":"This paper addresses the aspect of the meaningfulness of a national assessment in English as a foreign language, applying the fairness framework proposed by Kunnan (2018). We compared students’ performance on two receptive language skills of listening and reading on two subsequent national evaluations in Finland, taken by students at the end of compulsory basic education and at the end of general upper secondary education, respectively. The research questions focus on (1) the relationship between students’ performance on the two tests and their gender, language of schooling and parents’ educational level, and (2) the relationship between the students’ receptive language proficiency at the end of basic education and general upper secondary education. The data were analysed using linear regression and quantile regression analyses. The effect of background variables on the proficiency was stronger for low-performing students than for high-performing students. Moreover, students’ proficiency on the receptive skills at the end of basic education well predicted that at the end of general upper secondary education across several points in the score distribution. The findings also indicated persistent challenges with respect to educational equality and equity that requires ongoing attention by policy-makers and test designers.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":"132 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76424925","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Gebril, A. (Ed.) Learning-Oriented Language Assessment: Putting Theory into Practice. (编)《以学习为导向的语言评估:理论与实践》。
Q4 LINGUISTICS Pub Date : 2022-01-01 DOI: 10.58379/pbmv5990
A. Berger
n/a
N/A
{"title":"Gebril, A. (Ed.) Learning-Oriented Language Assessment: Putting Theory into Practice.","authors":"A. Berger","doi":"10.58379/pbmv5990","DOIUrl":"https://doi.org/10.58379/pbmv5990","url":null,"abstract":"<jats:p>n/a</jats:p>","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":"7 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73626737","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Implications of training university teachers in developing local writing rating scales 培训大学教师编制地方写作评定量表的意义
Q4 LINGUISTICS Pub Date : 2022-01-01 DOI: 10.58379/xvdf9070
O. Kvasova, Lyudmyla Hnapovska, V. Kalinichenko, Luliia Budas
Language assessment literacy is currently in search of new, modern conceptualisations in which contextual factors have a growing significance and impact (Tsagari, 2020). This article presents an initiative to promote writing assessment literacy in a culture-specific educational context. Assessment of writing belongs to the under-researched areas in Ukrainian higher education, wherein teachers have to act as raters and as rating scale developers without being properly trained in language assessment. The gaps in writing assessment literacy prompted research into the strengths and weaknesses of using a local rating scale developed by university teachers. It was conducted within an Erasmus + Staff mobility project in 2016-2019 and followed up by dissemination events held in several universities in Ukraine. The сurrent study aims to explore the impact of training in writing assessment on the processes and outcomes of university teachers’ development and use of analytic rating scales. The paper analyses how three teams of teachers from different universities coped with the task, and whether the training they underwent enabled them to design well-performing rating scales. The nine participants in the study developed three local context-specific analytic rating scales following the intuitive method of scale design, detailed in the guidelines prepared by the trainer. Given the same context (ESP) and the same CEFR level (B1 ->B2), we managed to compare the three local rating scales. The study testifies to a positive impact of the training on teachers’ literacy in writing assessment.
语言评估素养目前正在寻找新的、现代的概念,其中语境因素具有越来越大的意义和影响(Tsagari, 2020)。本文提出了在特定文化教育背景下促进写作评估素养的倡议。写作评估属于乌克兰高等教育研究不足的领域,其中教师必须充当评分员和评级量表开发人员,而没有经过适当的语言评估培训。写作评估能力的差距促使人们研究使用大学教师开发的本地评分量表的优缺点。它是在2016-2019年伊拉斯谟+员工流动项目中进行的,随后在乌克兰的几所大学举行了传播活动。本研究旨在探讨写作评估训练对大学教师发展过程和结果的影响,以及分析性评分量表的使用。本文分析了来自不同大学的三组教师是如何完成任务的,以及他们所接受的培训是否使他们能够设计出良好的评分量表。研究的九名参与者按照直观的量表设计方法,开发了三种当地具体情况的分析评定量表,具体内容见培训师编写的指导方针。给定相同的上下文(ESP)和相同的CEFR级别(B1 ->B2),我们设法比较了三种本地评级量表。本研究证明了培训对教师写作素养评估的积极影响。
{"title":"Implications of training university teachers in developing local writing rating scales","authors":"O. Kvasova, Lyudmyla Hnapovska, V. Kalinichenko, Luliia Budas","doi":"10.58379/xvdf9070","DOIUrl":"https://doi.org/10.58379/xvdf9070","url":null,"abstract":"Language assessment literacy is currently in search of new, modern conceptualisations in which contextual factors have a growing significance and impact (Tsagari, 2020). This article presents an initiative to promote writing assessment literacy in a culture-specific educational context. Assessment of writing belongs to the under-researched areas in Ukrainian higher education, wherein teachers have to act as raters and as rating scale developers without being properly trained in language assessment. The gaps in writing assessment literacy prompted research into the strengths and weaknesses of using a local rating scale developed by university teachers. It was conducted within an Erasmus + Staff mobility project in 2016-2019 and followed up by dissemination events held in several universities in Ukraine. The сurrent study aims to explore the impact of training in writing assessment on the processes and outcomes of university teachers’ development and use of analytic rating scales. The paper analyses how three teams of teachers from different universities coped with the task, and whether the training they underwent enabled them to design well-performing rating scales. The nine participants in the study developed three local context-specific analytic rating scales following the intuitive method of scale design, detailed in the guidelines prepared by the trainer. Given the same context (ESP) and the same CEFR level (B1 ->B2), we managed to compare the three local rating scales. The study testifies to a positive impact of the training on teachers’ literacy in writing assessment.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":"75 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88518060","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Language assessment literacy of language teachers in the context of adult education in Spain 西班牙成人教育背景下语言教师的语言评价素养
Q4 LINGUISTICS Pub Date : 2022-01-01 DOI: 10.58379/rrdv2344
Julia Consuelo Zabala Delgado, Cristina Rodriguez
Recent years have seen a growth in the use of language tests as measures of accountability within the education sector. In many countries, governmental institutions have promoted the involvement of teachers in language testing, providing training to boost the language assessment literacy (LAL) of teachers. This study aims to analyse the results of a large-scale effort to increase teachers’ LAL within the context of public language education for adults in Spain by shedding light on the scale and nature of teacher LAL, the impact of training as perceived by teachers, and their self-perceived further needs. Results show that, similarly to other countries in which teacher LAL has been studied, training in assessment is strongly influenced by contextual factors. Moreover, teachers perceive that this training has an impact not only on assessment-related tasks, but on their general teaching practice. Lastly, the findings reveal a significant correlation between the contents of assessment training courses and the teachers’ perception of further training needs. This could indicate that the more teachers learn about specific areas of language assessment, the more training in assessment they feel is needed, suggesting a gap in teachers’ awareness of their own LAL that materialises once training is provided.
近年来,越来越多的人使用语言测试来衡量教育部门的问责制。在许多国家,政府机构促进教师参与语言测试,提供培训以提高教师的语言评估素养。本研究旨在分析在西班牙成人公共语言教育背景下提高教师学习能力的大规模努力的结果,揭示教师学习能力的规模和性质,教师认为培训的影响,以及他们自我认为的进一步需求。结果表明,与研究教师学习能力的其他国家类似,评估培训受到情境因素的强烈影响。此外,教师认为这种培训不仅对评估相关的任务有影响,而且对他们的一般教学实践也有影响。最后,研究结果显示,评核培训课程的内容与教师对进一步培训需求的感知之间存在显著的相关关系。这可能表明,教师对语言评估的特定领域了解得越多,他们认为需要的评估培训就越多,这表明教师对自己的语言评估的认识存在差距,这种差距一旦提供培训就会显现出来。
{"title":"Language assessment literacy of language teachers in the context of adult education in Spain","authors":"Julia Consuelo Zabala Delgado, Cristina Rodriguez","doi":"10.58379/rrdv2344","DOIUrl":"https://doi.org/10.58379/rrdv2344","url":null,"abstract":"Recent years have seen a growth in the use of language tests as measures of accountability within the education sector. In many countries, governmental institutions have promoted the involvement of teachers in language testing, providing training to boost the language assessment literacy (LAL) of teachers. This study aims to analyse the results of a large-scale effort to increase teachers’ LAL within the context of public language education for adults in Spain by shedding light on the scale and nature of teacher LAL, the impact of training as perceived by teachers, and their self-perceived further needs. Results show that, similarly to other countries in which teacher LAL has been studied, training in assessment is strongly influenced by contextual factors. Moreover, teachers perceive that this training has an impact not only on assessment-related tasks, but on their general teaching practice. Lastly, the findings reveal a significant correlation between the contents of assessment training courses and the teachers’ perception of further training needs. This could indicate that the more teachers learn about specific areas of language assessment, the more training in assessment they feel is needed, suggesting a gap in teachers’ awareness of their own LAL that materialises once training is provided.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":"7 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81054209","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Contextual variables in written assessment feedback in a university-level Spanish program 大学水平西班牙语课程书面评估反馈中的上下文变量
Q4 LINGUISTICS Pub Date : 2022-01-01 DOI: 10.58379/aanz
Kathryn Hill, A. Ducasse
The ‘situated’ nature of assessment may help to explain why feedback interventions are successful in one setting but not in another. This study reanalyses data from an earlier study (Ducasse & Hill, 2019) using a coherent theory of context, Bronfenbrenner’s ecological framework (1979), to investigate contextual influences on teacher feedback practices and learner responses respectively. Participants comprised 15 beginner, intermediate and advanced level students in an Australian university Spanish language program. Data comprised summative feedback on writing tasks and audio-recordings and transcripts of teacher and student think-aloud protocols, and discussions. Data were analysed in relation to the five systems of Bronfenbrenner’s framework (microsystem, mesosystem, exosystem, macrosystem and chronosystem). The study found teacher practices appeared to be influenced by perceptions of the learners in addition to task and performance characteristics (microsystem), by institutional policies and practices (exosystem) and by external professional accreditation standards (macrosystem). Contextual factors found to influence learner responses included perceptions of the teacher, the timing and valence of feedback (microsystem), course level and maturity as a learner (chronosystem) as well as by other subjects the learners were enrolled in (mesosystem). The study demonstrates the utility of Bronfenbrenner’s framework for systematic reflection on contextual influences in language assessment research.
评估的“情境”性质可能有助于解释为什么反馈干预在一种环境中成功,而在另一种环境中却不成功。本研究使用连贯的语境理论,即Bronfenbrenner的生态框架(1979),重新分析了早期研究(Ducasse & Hill, 2019)的数据,分别研究了语境对教师反馈实践和学习者反应的影响。参与者包括澳大利亚一所大学西班牙语课程的15名初级、中级和高级学生。数据包括对写作任务的总结性反馈,以及教师和学生的有声思考协议和讨论的录音和成绩单。对Bronfenbrenner框架中的5个系统(微系统、中系统、外系统、宏观系统和时间系统)进行了数据分析。研究发现,除了任务和表现特征外,教师的做法似乎还受到学习者的看法(微系统)、制度政策和做法(外系统)和外部专业认证标准(宏观系统)的影响。影响学习者反应的环境因素包括对教师的感知、反馈的时间和效价(微系统)、课程水平和学习者的成熟度(时间系统)以及学习者所参加的其他科目(中系统)。本研究证明了布朗芬布伦纳的框架在语言评价研究中对语境影响的系统反思的实用性。
{"title":"Contextual variables in written assessment feedback in a university-level Spanish program","authors":"Kathryn Hill, A. Ducasse","doi":"10.58379/aanz","DOIUrl":"https://doi.org/10.58379/aanz","url":null,"abstract":"The ‘situated’ nature of assessment may help to explain why feedback interventions are successful in one setting but not in another. This study reanalyses data from an earlier study (Ducasse & Hill, 2019) using a coherent theory of context, Bronfenbrenner’s ecological framework (1979), to investigate contextual influences on teacher feedback practices and learner responses respectively. Participants comprised 15 beginner, intermediate and advanced level students in an Australian university Spanish language program. Data comprised summative feedback on writing tasks and audio-recordings and transcripts of teacher and student think-aloud protocols, and discussions. Data were analysed in relation to the five systems of Bronfenbrenner’s framework (microsystem, mesosystem, exosystem, macrosystem and chronosystem). The study found teacher practices appeared to be influenced by perceptions of the learners in addition to task and performance characteristics (microsystem), by institutional policies and practices (exosystem) and by external professional accreditation standards (macrosystem). Contextual factors found to influence learner responses included perceptions of the teacher, the timing and valence of feedback (microsystem), course level and maturity as a learner (chronosystem) as well as by other subjects the learners were enrolled in (mesosystem). The study demonstrates the utility of Bronfenbrenner’s framework for systematic reflection on contextual influences in language assessment research.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":"41 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72491304","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Language assessment literacy: Understanding the construct from Norwegian EFL teachers’ perspective 语言评价素养:从挪威英语教师的视角看语言评价素养的建构
Q4 LINGUISTICS Pub Date : 2022-01-01 DOI: 10.58379/unud5510
Henrik Bøhn, Dina Tsagari
The question of what language assessment literacy (LAL) entails continues to preoccupy the research community. In the current study empirical evidence from Norway was gathered in order to inform our understanding of the construct. The focus of the investigation was twofold: First, it explored how Norwegian EFL teachers understand LAL. Secondly, it examined to what extent the same teachers found the dimensions in Taylor’s (2013) LAL model relevant in the Norwegian context. To that end semi-structured interviews were carried out with 10 teachers at the lower and upper secondary levels in Norway. The results showed that the teachers attributed a lot of weight to disciplinary competence and formative assessment, and they agreed with Taylor on several of her hypothesized dimensions, such as the importance given to language pedagogy. However, they also disagreed with some of the other dimensions, such as test-specific skills (e.g., how to design and validate multiple-choice tests), which they believed to be less relevant in the Norwegian context. The findings point to the contextualized nature of teacher LAL and provide important knowledge on how teacher LAL can be conceptualized and how Taylor’s model can be developed and refined.
语言评估素养(LAL)的问题一直是研究界关注的焦点。在目前的研究中,收集了来自挪威的经验证据,以告知我们对该结构的理解。调查的重点有两个:首先,它探讨了挪威英语教师如何理解LAL。其次,它检查了相同的教师在多大程度上发现泰勒(2013)LAL模型中的维度与挪威背景相关。为此目的,对挪威的10名初中和高中教师进行了半结构化访谈。结果显示,教师们非常重视学科能力和形成性评估,他们同意泰勒在几个假设维度上的观点,比如对语言教学法的重视。然而,他们也不同意其他一些方面,例如特定于考试的技能(例如,如何设计和验证多项选择题考试),他们认为这与挪威的情况不太相关。这些发现指出了教师学习能力的情境化本质,并为如何将教师学习能力概念化以及如何发展和完善泰勒的模型提供了重要的知识。
{"title":"Language assessment literacy: Understanding the construct from Norwegian EFL teachers’ perspective","authors":"Henrik Bøhn, Dina Tsagari","doi":"10.58379/unud5510","DOIUrl":"https://doi.org/10.58379/unud5510","url":null,"abstract":"The question of what language assessment literacy (LAL) entails continues to preoccupy the research community. In the current study empirical evidence from Norway was gathered in order to inform our understanding of the construct. The focus of the investigation was twofold: First, it explored how Norwegian EFL teachers understand LAL. Secondly, it examined to what extent the same teachers found the dimensions in Taylor’s (2013) LAL model relevant in the Norwegian context. To that end semi-structured interviews were carried out with 10 teachers at the lower and upper secondary levels in Norway. The results showed that the teachers attributed a lot of weight to disciplinary competence and formative assessment, and they agreed with Taylor on several of her hypothesized dimensions, such as the importance given to language pedagogy. However, they also disagreed with some of the other dimensions, such as test-specific skills (e.g., how to design and validate multiple-choice tests), which they believed to be less relevant in the Norwegian context. The findings point to the contextualized nature of teacher LAL and provide important knowledge on how teacher LAL can be conceptualized and how Taylor’s model can be developed and refined.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":"203 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77405474","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Is the devil you know better? Testwiseness and eliciting evidence of interactional competence in familiar versus unfamiliar triadic speaking tasks 你更了解魔鬼吗?熟悉与不熟悉三合一口语任务中互动能力的测试性和引出性证据
Q4 LINGUISTICS Pub Date : 2022-01-01 DOI: 10.58379/ttfe6660
Nichola Glasson
Research has proven the impact testwiseness and test preparation can have on speaking test discourse (Lam, 2015; Luk, 2010; Hüttner, 2014). The question is perhaps not whether assessment leads to contrived interaction but how interactions may differ between a more familiar task (face-to-face Cambridge B2 First Speaking) and an entirely new online task. This conversation analytic case study of a group of three in two different tasks (one face-to-face, one online) sought to explore how testwiseness is implicated in candidate talk and to explicate differences in what a more familiar and a completely new task elicited. The analytic focus was on the conversational object “I agree” but includes reference to non-verbal behaviours, turn-taking practices and the relationship between agreement and topic shift. The findings indicate a contrast in terms of closure and progressivity – that is, an observable difference in the closure of topics and the forward movement of conversation – across the two tasks. The data illustrates how the same candidates performed very differently in two tasks in the space of 40 minutes. In so doing, it raises questions about broadening the potential assessments have to target interactional competence.
研究已经证明了测试性和考试准备对口语测试语篇的影响(Lam, 2015;陆,2010;Huttner, 2014)。问题可能不在于评估是否会导致人为的互动,而在于一个更熟悉的任务(面对面的剑桥B2口语)和一个全新的在线任务之间的互动有何不同。这个对话分析案例研究的对象是一组三人,他们参加了两项不同的任务(一项是面对面的,另一项是在线的),旨在探索测试性是如何影响候选人的谈话的,并解释一个更熟悉的任务和一个全新的任务所引发的差异。分析的重点是对话对象“我同意”,但也包括对非语言行为、轮流实践以及同意与话题转移之间关系的参考。研究结果表明,两项任务在结束性和进步性方面存在差异——也就是说,在话题结束和对话向前推进方面存在可观察到的差异。这些数据说明了同一位候选人在40分钟内完成两项任务时的表现有多么不同。在这样做的过程中,它提出了关于扩大潜在评估必须以相互作用能力为目标的问题。
{"title":"Is the devil you know better? Testwiseness and eliciting evidence of interactional competence in familiar versus unfamiliar triadic speaking tasks","authors":"Nichola Glasson","doi":"10.58379/ttfe6660","DOIUrl":"https://doi.org/10.58379/ttfe6660","url":null,"abstract":"Research has proven the impact testwiseness and test preparation can have on speaking test discourse (Lam, 2015; Luk, 2010; Hüttner, 2014). The question is perhaps not whether assessment leads to contrived interaction but how interactions may differ between a more familiar task (face-to-face Cambridge B2 First Speaking) and an entirely new online task. This conversation analytic case study of a group of three in two different tasks (one face-to-face, one online) sought to explore how testwiseness is implicated in candidate talk and to explicate differences in what a more familiar and a completely new task elicited. The analytic focus was on the conversational object “I agree” but includes reference to non-verbal behaviours, turn-taking practices and the relationship between agreement and topic shift. The findings indicate a contrast in terms of closure and progressivity – that is, an observable difference in the closure of topics and the forward movement of conversation – across the two tasks. The data illustrates how the same candidates performed very differently in two tasks in the space of 40 minutes. In so doing, it raises questions about broadening the potential assessments have to target interactional competence.","PeriodicalId":29650,"journal":{"name":"Studies in Language Assessment","volume":"46 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75409036","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Studies in Language Assessment
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1