Research Methods in Applied Linguistics最新文献_第2页

Examining refusal and acceptance sequences employing a data-driven binary rating approach 采用数据驱动的二元评级法研究拒绝和接受顺序

Research Methods in Applied Linguistics

Pub Date : 2024-11-05 DOI: 10.1016/j.rmal.2024.100161

Yunwen Su , Xi Chen

This paper presents the development and preliminary validation of a data-driven analytic binary rating rubric for assessing roleplay-elicited pragmatic performance of learners. The roleplay task consists of two items—one lunch invitation to elicit a refusal and one gift offer to elicit an acceptance in Chinese, both of which involve initiating proposals to return an earlier favor. Responses to return-to-favor invitations or offers in Chinese are challenging for L2 learners as they involve complicated discursive features. To create a practical, reliable tool for evaluating L2 performance, the researchers conducted a discourse analysis of L1 Chinese performance data (N = 22) to identify key discursive features in two domains— Intention (i.e., refusing the return-for-favor intention) and Decision (i.e., refusing/accepting the offer/invitation). A roleplay can receive up to 4 points in both domains, including one point each for orientation/directness, position, modification, and justification, for 8 total possible points. Scores assigned to both L1 and L2 performance according to the rubric reflected high inter-rater reliability. The effect of proficiency was significant on learner scores and interacted with rating domains and scenarios. Lower-level learners scored lower with addressing the interlocutor's return-for-favor intention than with the invitation or offer itself; they also scored lower with refusing than accepting an offer. By focusing on key discursive features rather than exhaustive interactional details, the rubric streamlines the evaluation process while maintaining accuracy and reliability. Future studies may extend the use of the rubric to other speech act sequences and languages and apply it to automated grading systems.

本文介绍了一种数据驱动的二元分析评分标准的开发和初步验证，用于评估学习者在角色扮演中的语用表现。角色扮演任务包括两个项目，一个是午餐邀请以引起拒绝，另一个是礼物提议以引起接受。对汉语学习者来说，回应 "还人情 "邀请或提议具有挑战性，因为它们涉及复杂的话语特征。为了创建一个实用、可靠的工具来评估 L2 的表现，研究人员对 L1 的汉语表现数据（N = 22）进行了话语分析，以确定两个领域中的关键话语特征--意图（即拒绝 "投桃报李 "的意图）和决定（即拒绝/接受提议/邀请）。一个角色扮演在两个领域中最多可得 4 分，其中定向/直接性、立场、修饰和理由各得 1 分，总共可得 8 分。根据评分标准给 L1 和 L2 表演打分反映出评分者之间的高度可靠性。熟练程度对学习者得分的影响很大，并且与评分领域和情景相互影响。水平较低的学习者在表达对话者的回报意图方面的得分低于表达邀请或提议本身的得分；他们在拒绝提议方面的得分也低于接受提议的得分。通过关注关键的话语特征而不是详尽的互动细节，该评分标准简化了评价过程，同时保持了准确性和可靠性。未来的研究可能会将该评分标准推广到其他言语行为序列和语言中，并将其应用到自动评分系统中。

{"title":"Examining refusal and acceptance sequences employing a data-driven binary rating approach","authors":"Yunwen Su , Xi Chen","doi":"10.1016/j.rmal.2024.100161","DOIUrl":"10.1016/j.rmal.2024.100161","url":null,"abstract":"<div><div>This paper presents the development and preliminary validation of a data-driven analytic binary rating rubric for assessing roleplay-elicited pragmatic performance of learners. The roleplay task consists of two items—one lunch invitation to elicit a refusal and one gift offer to elicit an acceptance in Chinese, both of which involve initiating proposals to return an earlier favor. Responses to return-to-favor invitations or offers in Chinese are challenging for L2 learners as they involve complicated discursive features. To create a practical, reliable tool for evaluating L2 performance, the researchers conducted a discourse analysis of L1 Chinese performance data (<em>N</em> = 22) to identify key discursive features in two domains— Intention (i.e., refusing the return-for-favor intention) and Decision (i.e., refusing/accepting the offer/invitation). A roleplay can receive up to 4 points in both domains, including one point each for orientation/directness, position, modification, and justification, for 8 total possible points. Scores assigned to both L1 and L2 performance according to the rubric reflected high inter-rater reliability. The effect of proficiency was significant on learner scores and interacted with rating domains and scenarios. Lower-level learners scored lower with addressing the interlocutor's return-for-favor intention than with the invitation or offer itself; they also scored lower with refusing than accepting an offer. By focusing on key discursive features rather than exhaustive interactional details, the rubric streamlines the evaluation process while maintaining accuracy and reliability. Future studies may extend the use of the rubric to other speech act sequences and languages and apply it to automated grading systems.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100161"},"PeriodicalIF":0.0,"publicationDate":"2024-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142593789","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Profiling learner development with a computerized dynamic assessment of a Japanese learner's L2 email writing 利用计算机动态评估日语学习者的第二语言电子邮件写作，剖析学习者的发展情况

Research Methods in Applied Linguistics

Pub Date : 2024-10-30 DOI: 10.1016/j.rmal.2024.100164

Allan Nicholas, John Blake

This study evaluates a computerized dynamic assessment (C-DA) method's potential to build a detailed pragmatics-focused developmental profile of a Japanese L2 English learner's L2 email writing. L2 email writing studies typically separate learning and assessment, using holistic scales to evaluate the pragmatic elements of learner texts. C-DA, grounded in sociocultural theory, unifies learning and assessment, administering email tasks with varying social contexts and providing immediate mediation within the learner's zone of proximal development (ZPD). Further, a diagnostic capacity allows for identification of specific pragmatics-related infelicities. The C-DA employs a dual-layered framework: an identification layer uses a coding scheme to automatically identify specific instances of perceived pragmatic inappropriateness, while the mediation layer provides ZPD-sensitive mediation. By evaluating the frequency and explicitness of mediation engaged in, in combination with qualitative examination of elicited email texts, the C-DA enables insights into learner development within the ZPD. We focus on an individual learner's interaction with the C-DA, evaluating the program's effectiveness in enabling a detailed learner diagnostic and developmental profile.

本研究评估了一种计算机化动态评估（C-DA）方法的潜力，该方法可为一名日语二级英语学习者的二级电子邮件写作建立以语用为重点的详细发展档案。第二语言电子邮件写作研究通常将学习和评估分开，使用整体量表来评估学习者文本的语用要素。以社会文化理论为基础的 C-DA 将学习和评估统一起来，在不同的社会背景下执行电子邮件任务，并在学习者的最近发展区（ZPD）内提供即时调解。此外，诊断功能还能识别与语用相关的具体缺陷。C-DA 采用双层框架：识别层使用编码方案自动识别感知到的语用不当的具体实例，而调解层则提供对 ZPD 敏感的调解。通过评估调解的频率和明确性，并结合对诱导电子邮件文本的定性检查，C-DA 能够深入了解学习者在 ZPD 中的发展情况。我们将重点放在单个学习者与 C-DA 的互动上，评估该程序在建立详细的学习者诊断和发展档案方面的有效性。

{"title":"Profiling learner development with a computerized dynamic assessment of a Japanese learner's L2 email writing","authors":"Allan Nicholas, John Blake","doi":"10.1016/j.rmal.2024.100164","DOIUrl":"10.1016/j.rmal.2024.100164","url":null,"abstract":"<div><div>This study evaluates a computerized dynamic assessment (C-DA) method's potential to build a detailed pragmatics-focused developmental profile of a Japanese L2 English learner's L2 email writing. L2 email writing studies typically separate learning and assessment, using holistic scales to evaluate the pragmatic elements of learner texts. C-DA, grounded in sociocultural theory, unifies learning and assessment, administering email tasks with varying social contexts and providing immediate mediation within the learner's zone of proximal development (ZPD). Further, a diagnostic capacity allows for identification of specific pragmatics-related infelicities. The C-DA employs a dual-layered framework: an identification layer uses a coding scheme to automatically identify specific instances of perceived pragmatic inappropriateness, while the mediation layer provides ZPD-sensitive mediation. By evaluating the frequency and explicitness of mediation engaged in, in combination with qualitative examination of elicited email texts, the C-DA enables insights into learner development within the ZPD. We focus on an individual learner's interaction with the C-DA, evaluating the program's effectiveness in enabling a detailed learner diagnostic and developmental profile.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100164"},"PeriodicalIF":0.0,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142540289","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Incidental vocabulary learning: A scientometric review 偶然词汇学习：科学计量学回顾

Research Methods in Applied Linguistics

Pub Date : 2024-10-22 DOI: 10.1016/j.rmal.2024.100160

Sofiya Shahiwala , D. R. Rahul , John R. Baker

Research on incidental vocabulary learning has gained increasing attention in language acquisition. With growing empirical evidence, traditional reviews have attempted to provide incidental vocabulary learning narrative and summary effects. However, a detailed analysis of publication trends and their dynamics is necessary to comprehensively understand the field's developments. Consequently, a scientometric review was conducted using 547 journal articles published between 2003 and 2023 from the Web of Science and Scopus databases. The analysis explored developments through three lenses: performance, document co-citation, and structural variation. Through performance analysis, prominent journals, affiliate universities, and authors who increasingly contributed to the field were identified. The document co-citation analysis revealed several major clusters, of which the top six were scrutinised. Findings highlighted recurrent themes such as the importance of topic familiarity, glosses, and input modality. Moreover, influential articles and their linkages were evaluated through structural variation analysis, and their key findings and citation bursts revealed frontier trends. Finally, limitations in empirical data were discussed to elicit future research directions. As an early attempt at a scientometric analysis of incidental vocabulary acquisition, these findings will be highly beneficial to language researchers, teachers, and teacher educators, not only for identifying potential research areas but also for adapting strategies for vocabulary instruction.

在语言习得领域，有关附带词汇学习的研究越来越受到关注。随着经验证据的不断增加，传统的综述试图提供附带词汇学习的叙述和总结效果。然而，为了全面了解该领域的发展，有必要对论文发表趋势及其动态进行详细分析。因此，我们利用科学网和 Scopus 数据库中 2003 年至 2023 年间发表的 547 篇期刊论文进行了科学计量学回顾。分析从三个角度探讨了该领域的发展：绩效、文献共引和结构变异。通过绩效分析，确定了对该领域贡献越来越大的著名期刊、附属大学和作者。文献共引分析揭示了几个主要群组，并对其中前六个群组进行了仔细研究。研究结果突出了一些重复出现的主题，如主题熟悉度、词汇和输入模式的重要性。此外，还通过结构变异分析对有影响力的文章及其联系进行了评估，这些文章的主要发现和被引频次揭示了前沿趋势。最后，对实证数据的局限性进行了讨论，以引出未来的研究方向。作为对附带词汇习得进行科学计量分析的早期尝试，这些发现将对语言研究人员、教师和教师教育者大有裨益，不仅有助于确定潜在的研究领域，还有助于调整词汇教学策略。

{"title":"Incidental vocabulary learning: A scientometric review","authors":"Sofiya Shahiwala , D. R. Rahul , John R. Baker","doi":"10.1016/j.rmal.2024.100160","DOIUrl":"10.1016/j.rmal.2024.100160","url":null,"abstract":"<div><div>Research on incidental vocabulary learning has gained increasing attention in language acquisition. With growing empirical evidence, traditional reviews have attempted to provide incidental vocabulary learning narrative and summary effects. However, a detailed analysis of publication trends and their dynamics is necessary to comprehensively understand the field's developments. Consequently, a scientometric review was conducted using 547 journal articles published between 2003 and 2023 from the Web of Science and Scopus databases. The analysis explored developments through three lenses: performance, document co-citation, and structural variation. Through performance analysis, prominent journals, affiliate universities, and authors who increasingly contributed to the field were identified. The document co-citation analysis revealed several major clusters, of which the top six were scrutinised. Findings highlighted recurrent themes such as the importance of topic familiarity, glosses, and input modality. Moreover, influential articles and their linkages were evaluated through structural variation analysis, and their key findings and citation bursts revealed frontier trends. Finally, limitations in empirical data were discussed to elicit future research directions. As an early attempt at a scientometric analysis of incidental vocabulary acquisition, these findings will be highly beneficial to language researchers, teachers, and teacher educators, not only for identifying potential research areas but also for adapting strategies for vocabulary instruction.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100160"},"PeriodicalIF":0.0,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142529689","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Ordinal response scales: Psychometric grounding for design and analysis 序数反应量表：设计和分析的心理基础

Research Methods in Applied Linguistics

Pub Date : 2024-10-21 DOI: 10.1016/j.rmal.2024.100156

Lukas Sönning

Ordinal response scales are commonly used in applied linguistics. To summarize the distribution of ratings or judgments provided by informants, these are usually converted into numbers and then averaged or analyzed with ordinary regression models. This approach has been criticized in the literature; one caveat (among others) is the assumption that distances between categories are known. The present paper illustrates how empirical insights into the perception of response labels may inform the design and analysis stage of a study. We start with a review of how ordinal scales are used in linguistic research. Our survey offers insights into typical scale layouts and analysis strategies, and it allows us to identify three commonly used rating dimensions (agreement, intensity, and frequency). We take stock of the experimental literature on the perception of relevant scale point labels and then demonstrate how psychometric insights may direct scale design and data analysis. This includes a careful consideration of measurement-theoretic and statistical issues surrounding the numeric-conversion approach to ordinal data. We focus on the consequences of these drawbacks for the interpretation of empirical findings, which will enable researchers to make informed decisions and avoid drawing false conclusions from their data. We present a case study on yous(e) in two varieties of English, which shows that reliance on psychometric scale values can alter statistical conclusions, while also giving due consideration to the key limitations of the numeric-conversion approach to ordinal data analysis.

顺序反应量表通常用于应用语言学。为了总结信息提供者提供的评分或判断的分布情况，通常将其转换成数字，然后求平均值或用普通回归模型进行分析。这种方法在文献中饱受批评；其中一个注意事项是假设类别之间的距离是已知的。本文阐述了如何通过经验来了解人们对回应标签的感知，从而为研究的设计和分析阶段提供参考。我们首先回顾了语言学研究中如何使用顺序量表。通过调查，我们了解了典型的量表布局和分析策略，并确定了三个常用的评分维度（同意度、强度和频率）。我们对相关量表点标签感知的实验文献进行了总结，然后展示了心理测量学的见解如何指导量表设计和数据分析。这包括对测量理论和统计问题的仔细考虑，这些问题围绕着对序数数据的数字转换方法。我们将重点放在这些弊端对实证研究结果解释的影响上，这将使研究人员能够做出明智的决定，避免从数据中得出错误的结论。我们以两种英语中的 yous(e)为案例，说明依赖心理测量量表值可能会改变统计结论，同时也适当考虑了数字转换法在序数数据分析中的主要局限性。

{"title":"Ordinal response scales: Psychometric grounding for design and analysis","authors":"Lukas Sönning","doi":"10.1016/j.rmal.2024.100156","DOIUrl":"10.1016/j.rmal.2024.100156","url":null,"abstract":"<div><div>Ordinal response scales are commonly used in applied linguistics. To summarize the distribution of ratings or judgments provided by informants, these are usually converted into numbers and then averaged or analyzed with ordinary regression models. This approach has been criticized in the literature; one caveat (among others) is the assumption that distances between categories are known. The present paper illustrates how empirical insights into the perception of response labels may inform the design and analysis stage of a study. We start with a review of how ordinal scales are used in linguistic research. Our survey offers insights into typical scale layouts and analysis strategies, and it allows us to identify three commonly used rating dimensions (agreement, intensity, and frequency). We take stock of the experimental literature on the perception of relevant scale point labels and then demonstrate how psychometric insights may direct scale design and data analysis. This includes a careful consideration of measurement-theoretic and statistical issues surrounding the numeric-conversion approach to ordinal data. We focus on the consequences of these drawbacks for the interpretation of empirical findings, which will enable researchers to make informed decisions and avoid drawing false conclusions from their data. We present a case study on <em>yous(e)</em> in two varieties of English, which shows that reliance on psychometric scale values can alter statistical conclusions, while also giving due consideration to the key limitations of the numeric-conversion approach to ordinal data analysis.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100156"},"PeriodicalIF":0.0,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142529688","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Developing and using an adapted mosaic approach to explore children's foreign language learning experiences in primary school 开发和使用经调整的马赛克方法，探索小学儿童的外语学习经验

Research Methods in Applied Linguistics

Pub Date : 2024-10-17 DOI: 10.1016/j.rmal.2024.100155

Victoria Schulze

This article describes an adapted mosaic approach, developed and used in a longitudinal, ethnographic study that explored children's foreign language learning (FLL) experiences in primary school in England. As a methodological innovation within the field of instructed early foreign language learning, the adapted approach addresses an identified need for a wider range of research methodologies and more inductive approaches. The original mosaic approach was initially considered to help address methodological questions that arose during the ethnography. That approach was subsequently adapted and developed in four main ways: firstly, for use within the longitudinal, ethnographic study; second, for use with its older, primary school-aged children; third, for development and use together with children in school rather than a tool designed for use on children and finally, where the focus was upon the process of completion rather than the end product. The adapted approach met the needs identified within the ethnography by further enhancing children's voices, their agency and perspectives within its data collection and analysis. Recommendations for its use in language learning contexts are made to support further research together with children, to help strengthen what is currently known and understood about children's experiences of instructed foreign language learning. This is based on the premise that knowing and understanding more about these will help inform future practices and initiatives.

本文介绍了一种经过改编的马赛克方法，该方法是在一项纵向人种学研究中开发和使用的，该研究探讨了英国小学儿童的外语学习（FLL）经历。作为早期外语学习指导领域的一项方法创新，经过调整的方法满足了对更广泛的研究方法和更多归纳方法的需求。最初考虑采用马赛克方法来帮助解决人种学研究过程中出现的方法问题。随后，该方法主要从四个方面进行了调整和发展：第一，用于纵向人种学研究；第二，用于年龄较大的小学学龄儿童；第三，与学校儿童共同开发和使用，而不是设计用于儿童的工具；最后，重点是完成过程而不是最终产品。经过调整的方法满足了人种学研究中确定的需求，在数据收集和分析过程中进一步增强了儿童的声音、儿童的能动性和儿童的视角。建议在语言学习环境中使用这种方法，以支持与儿童一起开展进一步研究，帮助加强目前对儿童外语教学经验的了解和认识。这样做的前提是，更多地了解和理解这些经验将有助于为未来的实践和行动提供依据。

{"title":"Developing and using an adapted mosaic approach to explore children's foreign language learning experiences in primary school","authors":"Victoria Schulze","doi":"10.1016/j.rmal.2024.100155","DOIUrl":"10.1016/j.rmal.2024.100155","url":null,"abstract":"<div><div>This article describes an adapted mosaic approach, developed and used in a longitudinal, ethnographic study that explored children's foreign language learning (FLL) experiences in primary school in England. As a methodological innovation within the field of instructed early foreign language learning, the adapted approach addresses an identified need for a wider range of research methodologies and more inductive approaches. The original mosaic approach was initially considered to help address methodological questions that arose during the ethnography. That approach was subsequently adapted and developed in four main ways: firstly, for use within the longitudinal, ethnographic study; second, for use with its older, primary school-aged children; third, for development and use <em>together with</em> children in school rather than a tool designed for use <em>on</em> children and finally, where the focus was upon the <em>process</em> of completion rather than the end product. The adapted approach met the needs identified within the ethnography by further enhancing children's voices, their agency and perspectives within its data collection and analysis. Recommendations for its use in language learning contexts are made to support further research together with children, to help strengthen what is currently known and understood about children's experiences of instructed foreign language learning. This is based on the premise that knowing and understanding more about these will help inform future practices and initiatives.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100155"},"PeriodicalIF":0.0,"publicationDate":"2024-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142445797","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Research methods for investigating young learners’ engagement with foreign languages outside of school 调查青少年学生在校外学习外语情况的研究方法

Research Methods in Applied Linguistics

Pub Date : 2024-10-13 DOI: 10.1016/j.rmal.2024.100159

Vanessa De Wilde

Recent studies have shown that L2 input learners engage with outside the classroom can play an important role in young L2 learners’ language development. This article discusses various methods and instruments to investigate extramural input that have been used with young learners (ages 6 to 12 years). Quantitative, qualitative, and mixed methods approaches to measure learners’ engagement with the L2 outside the classroom context are considered. The present article looks into similarities and differences between the instruments that have been used in previous studies and briefly touches upon promising methods that have not been used with young learners to date but could prove interesting for future studies. Some guidelines related to collecting data about extramural language activities with young learners are included. The article ends with some suggestions for adapting and/or improving existing instruments.

最近的研究表明，学习者在课堂外的第二语言输入对青少年第二语言学习者的语言发展起着重要作用。本文讨论了针对青少年学习者（6 至 12 岁）的各种校外输入调查方法和工具。文章考虑了定量、定性和混合方法，以衡量学习者在课堂外对 L2 的参与情况。本文探讨了以往研究中使用的工具之间的异同，并简要介绍了迄今为止尚未用于青少年学习者但可能对未来研究有意义的方法。本文还包括一些与收集青少年学生校外语言活动数据有关的指导原则。文章最后提出了一些调整和/或改进现有工具的建议。

引用次数: 0

Network science in a dynamic usage-based approach 基于动态使用方法的网络科学

Research Methods in Applied Linguistics

Pub Date : 2024-09-30 DOI: 10.1016/j.rmal.2024.100150

Susanne DeVore , Marjolijn Verspoor

In this study, we test the ability of network science to capture linguistic development over time in individuals from a dynamic usage-based perspective, a combination of a complex dynamic systems theory (CDST) approach and usage-based (UB) linguistics. Network science is designed to quantitatively analyze entire systems and captures complex interrelationships between components of those systems. We select network science measures that have potential to represent theoretically predicted individual language learning processes, specifically focusing on the development from prototype to schematized constructions at the macro- (network), micro- (word), and meso‑level (syntactic structure). To test this approach, we traced the beginning L2 Chinese development of two English speakers. The results are generally aligned to dynamic usage-based predictions in terms of prototypicality, schematization variability, and variation. Additionally, the network science approach allows us to identify a core set of words that emerges at the early stage of learning and is highly connected, seeming to drive development; it also suggests that variability plays different roles at different levels of analysis. Although it is preliminary to make strong conclusions about this set of words, it suggests that, as with other areas of inquiry, network science can reveal previously unidentified information about linguistic development and should be further explored.

在本研究中，我们测试了网络科学从基于动态使用的角度捕捉个体语言随时间发展的能力，这是复杂动态系统理论（CDST）方法和基于使用的语言学（UB）的结合。网络科学旨在对整个系统进行定量分析，并捕捉这些系统各组成部分之间复杂的相互关系。我们选择的网络科学测量方法有可能代表理论上预测的个人语言学习过程，特别是侧重于宏观（网络）、微观（单词）和中观（句法结构）层面从原型到图式化结构的发展。为了检验这种方法，我们追踪了两名英语使用者的初级 L2 中文发展过程。在原型性、图式化变异性和变异性方面，结果与基于动态用法的预测基本一致。此外，网络科学方法让我们确定了在学习早期阶段出现的核心词组，这些词组之间联系紧密，似乎推动着学习的发展；它还表明，变异性在不同的分析层次上发挥着不同的作用。尽管对这组词语做出强有力的结论尚属初步，但它表明，与其他研究领域一样，网络科学可以揭示以前未发现的有关语言发展的信息，因此应进一步加以探索。

{"title":"Network science in a dynamic usage-based approach","authors":"Susanne DeVore , Marjolijn Verspoor","doi":"10.1016/j.rmal.2024.100150","DOIUrl":"10.1016/j.rmal.2024.100150","url":null,"abstract":"<div><div>In this study, we test the ability of network science to capture linguistic development over time in individuals from a dynamic usage-based perspective, a combination of a complex dynamic systems theory (CDST) approach and usage-based (UB) linguistics. Network science is designed to quantitatively analyze entire systems and captures complex interrelationships between components of those systems. We select network science measures that have potential to represent theoretically predicted individual language learning processes, specifically focusing on the development from prototype to schematized constructions at the macro- (network), micro- (word), and meso‑level (syntactic structure). To test this approach, we traced the beginning L2 Chinese development of two English speakers. The results are generally aligned to dynamic usage-based predictions in terms of prototypicality, schematization variability, and variation. Additionally, the network science approach allows us to identify a core set of words that emerges at the early stage of learning and is highly connected, seeming to drive development; it also suggests that variability plays different roles at different levels of analysis. Although it is preliminary to make strong conclusions about this set of words, it suggests that, as with other areas of inquiry, network science can reveal previously unidentified information about linguistic development and should be further explored.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100150"},"PeriodicalIF":0.0,"publicationDate":"2024-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142358790","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A methodological exploration of the P-frame approach: Implications for developing phrase lists 对 P 框架方法的方法论探索：对编制短语表的影响

Research Methods in Applied Linguistics

Pub Date : 2024-09-19 DOI: 10.1016/j.rmal.2024.100154

Aysel Şahin Kızıl , Benet Vincent , Lee McCallum

The widespread availability of corpora and corpus tools has led to considerable advances in our understanding of phraseology and how it can be investigated. An increasingly influential approach to investigating phraseology is the retrieval and analysis of phrase frames (p-frames), recurrent word sequences with a variable slot. Although the p-frame approach is quite well-established, pedagogically oriented research in this area is still quite new, in particular the application of phrase frames to generate useful lists of phrases. This paper aims to contribute to methodological efforts in this direction by presenting a case study to illustrate decision-making throughout the process from corpus compilation to the generation of a final pedagogical list of phrases. Using a corpus of research article (RA) introductions in Health Sciences, this study shows how key decisions including p-frame length, frequency and range thresholds were arrived at. It also discusses exclusion criteria focusing on variability and predictability of p-frame fillers, and the fine-tuning of the resulting list, focusing on semantic coherence and pedagogical usefulness. Finally, we present the results of an initial evaluation of the list by stakeholders. The main contributions of the study to p-frame research methodology lie in the clarification of threshold settings, the potential contribution of variability and predictability to decision-making and the proposal of more in-depth concordance analysis of co-text to aid in the identification of phrases.

随着语料库和语料库工具的普及，我们对短语的理解和研究方法都有了长足的进步。短语框架（p-frames）是具有可变时隙的重复出现的单词序列，它是一种越来越有影响力的短语研究方法。虽然短语框架的方法已经相当成熟，但这一领域以教学为导向的研究仍然相当新颖，尤其是应用短语框架生成有用的短语列表。本文旨在通过一个案例研究，说明从语料汇编到生成最终教学短语表的整个过程中的决策过程，从而为这一方向的方法论研究做出贡献。本研究使用健康科学领域研究文章（RA）引言的语料库，展示了如何做出包括 p 帧长度、频率和范围阈值在内的关键决策。本研究还讨论了排除标准，重点是 p 框填充物的可变性和可预测性，以及对最终列表的微调，重点是语义连贯性和教学实用性。最后，我们介绍了利益相关者对清单的初步评估结果。本研究对 p-frame研究方法的主要贡献在于明确了阈值设置、可变性和可预测性对决策的潜在贡献，以及提出了对共同文本进行更深入的一致性分析以帮助识别短语的建议。

{"title":"A methodological exploration of the P-frame approach: Implications for developing phrase lists","authors":"Aysel Şahin Kızıl , Benet Vincent , Lee McCallum","doi":"10.1016/j.rmal.2024.100154","DOIUrl":"10.1016/j.rmal.2024.100154","url":null,"abstract":"<div><p>The widespread availability of corpora and corpus tools has led to considerable advances in our understanding of phraseology and how it can be investigated. An increasingly influential approach to investigating phraseology is the retrieval and analysis of phrase frames (p-frames), recurrent word sequences with a variable slot. Although the p-frame approach is quite well-established, pedagogically oriented research in this area is still quite new, in particular the application of phrase frames to generate useful lists of phrases. This paper aims to contribute to methodological efforts in this direction by presenting a case study to illustrate decision-making throughout the process from corpus compilation to the generation of a final pedagogical list of phrases. Using a corpus of research article (RA) introductions in Health Sciences, this study shows how key decisions including p-frame length, frequency and range thresholds were arrived at. It also discusses exclusion criteria focusing on variability and predictability of p-frame fillers, and the fine-tuning of the resulting list, focusing on semantic coherence and pedagogical usefulness. Finally, we present the results of an initial evaluation of the list by stakeholders. The main contributions of the study to p-frame research methodology lie in the clarification of threshold settings, the potential contribution of variability and predictability to decision-making and the proposal of more in-depth concordance analysis of co-text to aid in the identification of phrases.</p></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100154"},"PeriodicalIF":0.0,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142270721","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Building custom NLP tools to annotate discourse-functional features for second language writing research: A tutorial 为第二语言写作研究建立自定义 NLP 工具以标注话语功能特征：教程

Research Methods in Applied Linguistics

Pub Date : 2024-09-18 DOI: 10.1016/j.rmal.2024.100153

Masaki Eguchi , Kristopher Kyle

The current tutorial paper describes a process of developing a custom natural language processing model with a particular focus on a discourse annotation task. After an overview of recent developments in natural language processing (NLP), the paper discusses the development of the Engagement Analyzer (Eguchi & Kyle, 2023), focusing on corpus annotation, the machine learning model, model training, evaluation, and dissemination. A step-by-step tutorial of this process via the spaCy Python package is provided. The paper highlights the feasibility of developing custom NLP tools to enhance the scalability and replicability of the annotation of context-sensitive linguistic features in L2 writing research.

本教程论文介绍了开发定制自然语言处理模型的过程，尤其侧重于话语注释任务。在概述了自然语言处理（NLP）的最新发展之后，本文讨论了 Engagement Analyzer（Eguchi & Kyle, 2023）的开发过程，重点是语料标注、机器学习模型、模型训练、评估和传播。本文通过 spaCy Python 软件包提供了该过程的分步教程。本文强调了开发定制 NLP 工具的可行性，以提高 L2 写作研究中语境敏感语言特征标注的可扩展性和可复制性。

引用次数: 0

Moving beyond binary language status in research: Investigating early foreign language learning and linguistic distance 在研究中超越二元语言地位：调查早期外语学习和语言距离

Research Methods in Applied Linguistics

Pub Date : 2024-09-09 DOI: 10.1016/j.rmal.2024.100151

Nils Jaekel , Michael Schurig , Eliane Lorenz

Globalization and migration continue to shape our societies, including educational contexts such as school classrooms. In response to young learners’ linguistic needs, particularly in the context of foreign language learning in Germany, educational approaches need to be adapted to meet the needs of multilingual students. Current, binary approaches accounting for diverse linguistic backgrounds of students in research assume a high degree of homogeneity among multilingual students. Linguistic distance measures may provide alternative, more fine-grained, continuous tools to account for linguistic diversity. This study employs lexical linguistic distance to account for young language learners’ linguistic diversity in a reanalysis of Jaekel et al. (2017). Additionally, mixed-effects modeling was employed to factor in within-class effects for within-class factors versus structural equation modeling, which was previously used. The results outline that linguistic distance provides additional information beyond binary language status. Mixed effects modeling renders comparable results with the same tendencies, but yields more nuanced perspectives on the data.

全球化和移民继续影响着我们的社会，包括学校课堂等教育环境。为了满足年轻学习者的语言需求，特别是在德国的外语学习环境中，需要调整教育方法，以满足多语种学生的需求。目前，在研究中考虑学生不同语言背景的二元方法假定了多语言学生之间的高度同质性。语言距离测量法可以提供替代性的、更精细的、连续的工具，来说明语言的多样性。本研究在对 Jaekel 等人（2017）的研究进行重新分析时，采用了词汇语言距离来解释年轻语言学习者的语言多样性。此外，与之前使用的结构方程模型相比，本研究采用了混合效应模型来考虑班内因素的班内效应。结果表明，语言距离提供了二元语言状态之外的额外信息。混合效应模型得出了具有相同倾向的可比结果，但对数据产生了更细微的影响。

{"title":"Moving beyond binary language status in research: Investigating early foreign language learning and linguistic distance","authors":"Nils Jaekel , Michael Schurig , Eliane Lorenz","doi":"10.1016/j.rmal.2024.100151","DOIUrl":"10.1016/j.rmal.2024.100151","url":null,"abstract":"<div><p>Globalization and migration continue to shape our societies, including educational contexts such as school classrooms. In response to young learners’ linguistic needs, particularly in the context of foreign language learning in Germany, educational approaches need to be adapted to meet the needs of multilingual students. Current, binary approaches accounting for diverse linguistic backgrounds of students in research assume a high degree of homogeneity among multilingual students. Linguistic distance measures may provide alternative, more fine-grained, continuous tools to account for linguistic diversity. This study employs lexical linguistic distance to account for young language learners’ linguistic diversity in a reanalysis of Jaekel et al. (2017). Additionally, mixed-effects modeling was employed to factor in within-class effects for within-class factors versus structural equation modeling, which was previously used. The results outline that linguistic distance provides additional information beyond binary language status. Mixed effects modeling renders comparable results with the same tendencies, but yields more nuanced perspectives on the data.</p></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"3 3","pages":"Article 100151"},"PeriodicalIF":0.0,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2772766124000570/pdfft?md5=9492363d23de861a48c0f3725af5df5f&pid=1-s2.0-S2772766124000570-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142162284","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0