Research Methods in Applied Linguistics最新文献_第9页

A systematic examination of generative artificial intelligence (GenAI) use guidelines in applied linguistics journals 应用语言学期刊中生成式人工智能（GenAI）使用指南的系统研究

Research Methods in Applied Linguistics

Pub Date : 2025-06-20 DOI: 10.1016/j.rmal.2025.100227

Shuhui Yin, Carol A. Chapelle

The unannounced appearance of GenAI in 2022 and the speed of its adoption by researchers have left many questions unanswered about its accepted ethical use, with no apparent consensus among applied linguists. In this context, it’s essential for researchers to develop their GenAI literacy for research to engage with GenAI effectively and responsibly. This study contributes to identifying key components of this literacy through examining accepted GenAI uses in research practices. Based on a systematically sampled collection of 170 high-impact journals in applied linguistics, we investigated the scope and nature of GenAI use guidelines provided by 76 journals intended to guide authors. A checklist including four items regarding general statements and 17 items regarding three categories of specific aspects that GenAI guidelines target (authorship, uses, and human responsibility) was identified. Our findings reveal that (1) less than half of the journals provided GenAI use guidelines to guide authors, (2) the number of specific aspects varied across journals, with most falling short of comprehensive coverage, and (3) disagreements were observed about whether AI can be cited and used for manuscript drafting, idea generating, image generating, data generation, data collection, and data analysis and interpretation. Additionally, journals varied in their guidance on how to disclose GenAI uses. We propose recommendations for journals in improving their AI guidelines. Importantly, we introduce and conceptualize the new construct GenAI literacy for research article writing (GenAI-LR) that is important for authors to develop. We provide actionable recommendations accordingly based on our findings.

GenAI在2022年的突然出现，以及它被研究人员采用的速度，留下了许多关于它被接受的道德用途的悬而未决的问题，应用语言学家之间没有明显的共识。在这种背景下，研究人员必须培养他们的GenAI知识，以便有效和负责任地参与研究。本研究通过检查在研究实践中公认的GenAI使用，有助于确定这种素养的关键组成部分。基于系统抽样收集的170种应用语言学高影响力期刊，我们调查了76种旨在指导作者的期刊提供的GenAI使用指南的范围和性质。确定了一份清单，其中包括关于一般陈述的四项和关于GenAI指南目标（作者身份、用途和人类责任）的三类具体方面的17项。我们的研究结果显示：(1)不到一半的期刊提供了GenAI使用指南来指导作者；(2)不同期刊在具体方面的数量各不相同，大多数没有全面覆盖；(3)关于AI是否可以被引用并用于手稿起草、想法生成、图像生成、数据生成、数据收集以及数据分析和解释方面存在分歧。此外，各期刊对如何披露GenAI用途的指导也各不相同。我们为期刊提出了改进其人工智能指南的建议。重要的是，我们引入并概念化了研究文章写作的基因素养（GenAI- lr）的新结构，这对作者的发展很重要。我们根据调查结果提供相应的可行建议。

{"title":"A systematic examination of generative artificial intelligence (GenAI) use guidelines in applied linguistics journals","authors":"Shuhui Yin, Carol A. Chapelle","doi":"10.1016/j.rmal.2025.100227","DOIUrl":"10.1016/j.rmal.2025.100227","url":null,"abstract":"<div><div>The unannounced appearance of GenAI in 2022 and the speed of its adoption by researchers have left many questions unanswered about its accepted ethical use, with no apparent consensus among applied linguists. In this context, it’s essential for researchers to develop their GenAI literacy for research to engage with GenAI effectively and responsibly. This study contributes to identifying key components of this literacy through examining accepted GenAI uses in research practices. Based on a systematically sampled collection of 170 high-impact journals in applied linguistics, we investigated the scope and nature of GenAI use guidelines provided by 76 journals intended to guide authors. A checklist including four items regarding general statements and 17 items regarding three categories of specific aspects that GenAI guidelines target (authorship, uses, and human responsibility) was identified. Our findings reveal that (1) less than half of the journals provided GenAI use guidelines to guide authors, (2) the number of specific aspects varied across journals, with most falling short of comprehensive coverage, and (3) disagreements were observed about whether AI can be cited and used for manuscript drafting, idea generating, image generating, data generation, data collection, and data analysis and interpretation. Additionally, journals varied in their guidance on how to disclose GenAI uses. We propose recommendations for journals in improving their AI guidelines. Importantly, we introduce and conceptualize the new construct GenAI literacy for research article writing (GenAI-LR) that is important for authors to develop. We provide actionable recommendations accordingly based on our findings.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 3","pages":"Article 100227"},"PeriodicalIF":0.0,"publicationDate":"2025-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144322641","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Evaluating the validity of web-based reaction-time tasks for assessing L2 grammatical knowledge in young learners 评估基于网络的反应时间任务评估年轻学习者第二语言语法知识的有效性

Research Methods in Applied Linguistics

Pub Date : 2025-06-20 DOI: 10.1016/j.rmal.2025.100228

Takeshi Ishihara , Akira Hamada

This study investigated the validity of web-based psycholinguistic tasks for assessing automatized explicit and implicit grammatical knowledge in young second language (L2) learners. Specifically, it examined whether a time-pressured grammaticality judgment task and a self-paced reading task administered online produce results comparable to those obtained in an in-person setting. In addition, the influence of the Big Five personality traits on task participation and performance was explored. A total of 192 Japanese first-year middle school students were assigned to either a School group (in-class setting) or a Home group (online setting). The results revealed that accuracy performance on both tasks was generally comparable across settings, supporting the feasibility of remote task administration. However, reaction-time-based measures were more sensitive to testing conditions. In the grammaticality judgment task, the expected reaction-time effect (faster responses to grammatical items) was observed in the School group but not in the Home group. Similarly, the Home group produced faster and more variable reading times in the self-paced reading task, and the expected slowdown for grammatical errors was not observed, raising concerns about the validity of online reading-time data. Personality traits, particularly neuroticism, conscientiousness, and agreeableness, were associated with task participation and reaction-time variability, highlighting the role of individual differences in online task behavior. These findings demonstrate the need for caution when using reaction-time measures in unsupervised web-based experiments with young L2 learners and offer practical recommendations for enhancing data quality in remote L2 assessment.

本研究调查了基于网络的心理语言学任务在评估年轻第二语言学习者自动化显性和隐性语法知识方面的有效性。具体来说，该研究检查了在线执行的时间压力语法判断任务和自定节奏阅读任务是否会产生与面对面环境中获得的结果相当的结果。此外，还探讨了大五人格特质对任务参与和绩效的影响。共有192名日本一年级中学生被分配到学校组（课堂设置）和家庭组（在线设置）。结果显示，在不同设置下，这两种任务的准确性表现大致相当，这支持了远程任务管理的可行性。然而，基于反应时间的测量对测试条件更敏感。在语法判断任务中，“学校”组存在预期的反应时间效应（对语法问题的反应速度更快），而“家”组不存在预期的反应时间效应。同样，Home组在自定节奏阅读任务中产生了更快和更可变的阅读时间，并且没有观察到预期的语法错误减慢，这引起了对在线阅读时间数据有效性的担忧。人格特征，特别是神经质、尽责性和亲和性，与任务参与和反应时间变异性有关，突出了个体差异在在线任务行为中的作用。这些发现表明，在对年轻的二语学习者进行无监督的网络实验时，使用反应时间测量需要谨慎，并为提高远程二语评估的数据质量提供了实用建议。

{"title":"Evaluating the validity of web-based reaction-time tasks for assessing L2 grammatical knowledge in young learners","authors":"Takeshi Ishihara , Akira Hamada","doi":"10.1016/j.rmal.2025.100228","DOIUrl":"10.1016/j.rmal.2025.100228","url":null,"abstract":"<div><div>This study investigated the validity of web-based psycholinguistic tasks for assessing automatized explicit and implicit grammatical knowledge in young second language (L2) learners. Specifically, it examined whether a time-pressured grammaticality judgment task and a self-paced reading task administered online produce results comparable to those obtained in an in-person setting. In addition, the influence of the Big Five personality traits on task participation and performance was explored. A total of 192 Japanese first-year middle school students were assigned to either a School group (in-class setting) or a Home group (online setting). The results revealed that accuracy performance on both tasks was generally comparable across settings, supporting the feasibility of remote task administration. However, reaction-time-based measures were more sensitive to testing conditions. In the grammaticality judgment task, the expected reaction-time effect (faster responses to grammatical items) was observed in the School group but not in the Home group. Similarly, the Home group produced faster and more variable reading times in the self-paced reading task, and the expected slowdown for grammatical errors was not observed, raising concerns about the validity of online reading-time data. Personality traits, particularly neuroticism, conscientiousness, and agreeableness, were associated with task participation and reaction-time variability, highlighting the role of individual differences in online task behavior. These findings demonstrate the need for caution when using reaction-time measures in unsupervised web-based experiments with young L2 learners and offer practical recommendations for enhancing data quality in remote L2 assessment.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 3","pages":"Article 100228"},"PeriodicalIF":0.0,"publicationDate":"2025-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144330206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Challenging lexical coverage conventions: Evaluating the vocabulary demands of family-genre film and television 挑战词汇覆盖惯例：评估家庭类型电影和电视的词汇需求

Research Methods in Applied Linguistics

Pub Date : 2025-06-19 DOI: 10.1016/j.rmal.2025.100230

Brett Milliner , Geoffrey Pinchbeck

The contribution of studies investigating lexical coverage to the field of applied linguistics cannot be understated. Lexical coverage research has helped establish the vocabulary knowledge most essential for second language (L2) comprehension and elevate the importance of high-frequency vocabulary knowledge acquisition. Approaches to lexical coverage research have, however, begun to come under closer scrutiny in recent studies, with some experts questioning the accuracy of coverage estimates. Understanding these limitations, the current study applies an alternative approach to evaluating the lexical knowledge required to comprehend the OPUS-family-genre corpus, a collection of closed captions from 1597 family-genre films and television programs (10,744,767 tokens). In contrast to previous conventions that used band-based (1000-word) predictions of lexical coverage, in this study, coverage is evaluated at the individual word-unit level. It compares the coverage provided by four word lists: (1) a lemma list derived from tagging the OPUS-family-genre corpus, (2) a flemma list, and two word-family lists, (3) the BNC, and (4) the BNC/COCA. The study also models how a part-of-speech lexical tagger (TagAnt) can be used to evaluate lemma-based lexical coverage. The analysis revealed that English language learners will know 90, 95, and 98% of the running words appearing in family-genre films and television if they know the first 855, 2005, and 4393 flemmas, from the attached word lists. More simply, knowing the first 900 words from our supplementary word frequency lists would enable English language learners to start viewing family-genre films and television.

词汇覆盖研究对应用语言学领域的贡献是不可低估的。词汇覆盖研究有助于确立对第二语言理解至关重要的词汇知识，提高高频词汇知识习得的重要性。然而，在最近的研究中，词汇覆盖研究的方法开始受到更严格的审查，一些专家质疑覆盖估计的准确性。了解了这些局限性，本研究采用了另一种方法来评估理解opus家庭类型语料库所需的词汇知识，该语料库收集了来自1597部家庭类型电影和电视节目（10,744,767个符号）的封闭字幕。与以前使用基于频带（1000字）的词汇覆盖率预测的惯例相反，在本研究中，覆盖率是在单个单词单位级别进行评估的。它比较了四个词表提供的覆盖范围：(1)从标注OPUS-family-genre语料库派生的引理表，(2)flemma表和两个词族表，(3)BNC和(4)BNC/COCA。该研究还对词性词汇标注器（TagAnt）如何用于评估基于引理的词汇覆盖进行了建模。分析显示，如果英语学习者知道了家庭类型电影和电视中出现的前855、2005和4393部电影的单词表，他们就能掌握其中的90%、95%和98%。更简单地说，从我们的补充词频表中了解前900个单词将使英语学习者能够开始观看家庭类型的电影和电视。

{"title":"Challenging lexical coverage conventions: Evaluating the vocabulary demands of family-genre film and television","authors":"Brett Milliner , Geoffrey Pinchbeck","doi":"10.1016/j.rmal.2025.100230","DOIUrl":"10.1016/j.rmal.2025.100230","url":null,"abstract":"<div><div>The contribution of studies investigating lexical coverage to the field of applied linguistics cannot be understated. Lexical coverage research has helped establish the vocabulary knowledge most essential for second language (L2) comprehension and elevate the importance of high-frequency vocabulary knowledge acquisition. Approaches to lexical coverage research have, however, begun to come under closer scrutiny in recent studies, with some experts questioning the accuracy of coverage estimates. Understanding these limitations, the current study applies an alternative approach to evaluating the lexical knowledge required to comprehend the OPUS-family-genre corpus, a collection of closed captions from 1597 family-genre films and television programs (10,744,767 tokens). In contrast to previous conventions that used band-based (1000-word) predictions of lexical coverage, in this study, coverage is evaluated at the individual word-unit level. It compares the coverage provided by four word lists: (1) a lemma list derived from tagging the OPUS-family-genre corpus, (2) a flemma list, and two word-family lists, (3) the BNC, and (4) the BNC/COCA. The study also models how a part-of-speech lexical tagger (TagAnt) can be used to evaluate lemma-based lexical coverage. The analysis revealed that English language learners will know 90, 95, and 98% of the running words appearing in family-genre films and television if they know the first 855, 2005, and 4393 flemmas, from the attached word lists. More simply, knowing the first 900 words from our supplementary word frequency lists would enable English language learners to start viewing family-genre films and television.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 3","pages":"Article 100230"},"PeriodicalIF":0.0,"publicationDate":"2025-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144322640","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Unique and shared roles of the LLAMA subtests for prediction of initial L2 achievement: An application of regression commonality analysis LLAMA子测试在预测初始L2成就中的独特和共同作用：回归共性分析的应用

Research Methods in Applied Linguistics

Pub Date : 2025-06-19 DOI: 10.1016/j.rmal.2025.100224

Philip S. Dale , Lars Bokander , Richard L. Sparks

Little research has examined the relations of the LLAMA subtests beyond predictive correlations and simple regressions. In this secondary analysis of data from Bokander (2020), we use regression commonality analyses (RCA) to address multicollinearity by decomposing the LLAMA predictive variance into unique components for each subtest alone and for each possible subtest combination. Fifty-five students with Germanic L1 backgrounds completed the LLAMA, followed by an introductory Swedish course, and then a written C-test. LLAMA-D, sound-sequence recognition, was the most important unique predictor of L2 achievement. LLAMA-E (sound-symbol association) unique variance and shared variance with LLAMA-D and LLAMA-B (vocabulary learning) was the next most important contributor to prediction. Similar to results for MLAT, these results demonstrate the major role of phonetic script/sound-symbol relationship skills both uniquely and shared with other subtests. The most important difference is the equally important, distinct role of speech sound-sequence recognition, a skill not previously included in aptitude tests prior to the LLAMA. The paper concludes with a discussion of the strengths and limitations of regression commonality analysis, which appears to have considerable usefulness for studies involving prediction.

除了预测相关性和简单回归之外，很少有研究检验LLAMA子测试之间的关系。在对Bokander（2020）数据的二次分析中，我们使用回归共性分析（RCA）通过将LLAMA预测方差分解为单独的每个子测试和每个可能的子测试组合的唯一组件来解决多重共线性问题。55名拥有日耳曼L1背景的学生完成了LLAMA，随后是瑞典语入门课程，然后是书面c测试。LLAMA-D，声音序列识别，是第二语言成就最重要的唯一预测因子。LLAMA-E（音-符号关联）的独特方差和与LLAMA-D和LLAMA-B（词汇学习）的共享方差是预测的第二重要因素。与MLAT的结果类似，这些结果证明了音标/音-符号关系技能在其他子测试中独特和共享的主要作用。最重要的区别是语音序列识别同样重要，具有独特的作用，这是在LLAMA之前的能力倾向测试中没有包括的技能。本文最后讨论了回归共性分析的优势和局限性，它似乎对涉及预测的研究有相当大的用处。

{"title":"Unique and shared roles of the LLAMA subtests for prediction of initial L2 achievement: An application of regression commonality analysis","authors":"Philip S. Dale , Lars Bokander , Richard L. Sparks","doi":"10.1016/j.rmal.2025.100224","DOIUrl":"10.1016/j.rmal.2025.100224","url":null,"abstract":"<div><div>Little research has examined the relations of the LLAMA subtests beyond predictive correlations and simple regressions. In this secondary analysis of data from Bokander (2020), we use regression commonality analyses (RCA) to address multicollinearity by decomposing the LLAMA predictive variance into unique components for each subtest alone and for each possible subtest combination. Fifty-five students with Germanic L1 backgrounds completed the LLAMA, followed by an introductory Swedish course, and then a written C-test. LLAMA-D, sound-sequence recognition, was the most important unique predictor of L2 achievement. LLAMA-E (sound-symbol association) unique variance and shared variance with LLAMA-D and LLAMA-B (vocabulary learning) was the next most important contributor to prediction. Similar to results for MLAT, these results demonstrate the major role of phonetic script/sound-symbol relationship skills both uniquely and shared with other subtests. The most important difference is the equally important, distinct role of speech sound-sequence recognition, a skill not previously included in aptitude tests prior to the LLAMA. The paper concludes with a discussion of the strengths and limitations of regression commonality analysis, which appears to have considerable usefulness for studies involving prediction.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 3","pages":"Article 100224"},"PeriodicalIF":0.0,"publicationDate":"2025-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144322639","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Rethinking fluency in task-based L2 writing: A critical examination of product- and process-based measures 重新思考基于任务的第二语言写作的流畅性：对基于产品和过程的衡量标准的批判性检查

Research Methods in Applied Linguistics

Pub Date : 2025-06-14 DOI: 10.1016/j.rmal.2025.100225

Mahmoud Abdi Tabari , Mark D. Johnson

This paper offers a theoretically informed critical examination of current conceptualizations of fluency in task-based writing, a growing focus within second and foreign language writing research and pedagogy. To date, research in this area has employed both product- and process-oriented metrics, with a notable shift toward process-based approaches over the past few years. This review traces the evolution of these analytic frameworks in writing research, evaluating their respective strengths and limitations. Building on these insights, the paper provides recommendations for the application of each approach in pedagogical and research settings, emphasizing the importance of inclusivity and equity in both second and foreign language instruction and task-based L2 writing research.

本文对当前任务型写作中流畅性的概念进行了理论分析，这是第二语言和外语写作研究和教学中日益关注的问题。到目前为止，这一领域的研究已经采用了面向产品和面向过程的度量标准，在过去的几年中，有一个明显的转向基于过程的方法。本文回顾了这些分析框架在写作研究中的演变，评估了它们各自的优势和局限性。基于这些见解，本文为每种方法在教学和研究环境中的应用提供了建议，强调了在第二语言和外语教学以及基于任务的第二语言写作研究中包容性和公平性的重要性。

引用次数: 0

Conducting sociolinguistic interviews via generative AI: A methods tutorial 通过生成人工智能进行社会语言学访谈：方法教程

Research Methods in Applied Linguistics

Pub Date : 2025-06-11 DOI: 10.1016/j.rmal.2025.100223

Annita Stell, Hao Tran, Peter Crosthwaite

Sociolinguistic interviews (Labov, 1984) are integral to dialectology studies, providing insights into language variation and the social contexts influencing the emergence of new dialects (Hoffman, 2013; Pizarro Pedraza, 2016). Such data, while incredibly valuable, is typically time-consuming and expensive to collect. However, with the advent of generative AI (GenAI) applications e.g., ChatGPT 4o purported to produce discourse in multiple languages, its affordances for generating sociolinguistic interview data across different dialects of dialect-rich languages remain largely unknown. Building on a Gen-AI-assisted Mandarin/ Vietnamese dialect study in Tran and Stell (2024), this tutorial offers step-by-step guidance on conducting sociolinguistic interviews with ChatGPT. Key steps include generating prompts by establishing a dialogue context, formulating appropriate structured open-ended questions to elicit target dialectal varieties, and a cross-validation process with actual dialect speakers. While acknowledging the potential limitations of conducting sociolinguistic interviews with ChatGPT, this tutorial serves to aid those new to GenAI for dialectology research while suggesting future directions for refining this process as GenAI continues to develop.

社会语言学访谈（Labov, 1984）是方言学研究不可或缺的一部分，它提供了对语言变化和影响新方言出现的社会背景的见解(Hoffman, 2013；Pizarro Pedraza, 2016)。这些数据虽然非常有价值，但收集起来通常既耗时又昂贵。然而，随着生成式人工智能（GenAI）应用的出现，例如ChatGPT 40声称可以生成多种语言的话语，它在跨方言丰富语言的不同方言生成社会语言学访谈数据方面的功能仍然很大程度上未知。基于gen - ai辅助的Tran和Stell普通话/越南方言研究（2024），本教程提供了使用ChatGPT进行社会语言学访谈的逐步指导。关键步骤包括通过建立对话上下文来生成提示，制定适当的结构化开放式问题以引出目标方言品种，以及与实际方言使用者进行交叉验证过程。虽然承认使用ChatGPT进行社会语言学访谈的潜在局限性，但本教程旨在帮助那些刚接触GenAI的人进行方言研究，同时建议随着GenAI的不断发展，改进这一过程的未来方向。

{"title":"Conducting sociolinguistic interviews via generative AI: A methods tutorial","authors":"Annita Stell, Hao Tran, Peter Crosthwaite","doi":"10.1016/j.rmal.2025.100223","DOIUrl":"10.1016/j.rmal.2025.100223","url":null,"abstract":"<div><div>Sociolinguistic interviews (Labov, 1984) are integral to dialectology studies, providing insights into language variation and the social contexts influencing the emergence of new dialects (Hoffman, 2013; Pizarro Pedraza, 2016). Such data, while incredibly valuable, is typically time-consuming and expensive to collect. However, with the advent of generative AI (GenAI) applications e.g., ChatGPT 4o purported to produce discourse in multiple languages, its affordances for generating sociolinguistic interview data across different dialects of dialect-rich languages remain largely unknown. Building on a Gen-AI-assisted Mandarin/ Vietnamese dialect study in Tran and Stell (2024), this tutorial offers step-by-step guidance on conducting sociolinguistic interviews with ChatGPT. Key steps include generating prompts by establishing a dialogue context, formulating appropriate structured open-ended questions to elicit target dialectal varieties, and a cross-validation process with actual dialect speakers. While acknowledging the potential limitations of conducting sociolinguistic interviews with ChatGPT, this tutorial serves to aid those new to GenAI for dialectology research while suggesting future directions for refining this process as GenAI continues to develop.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 3","pages":"Article 100223"},"PeriodicalIF":0.0,"publicationDate":"2025-06-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144253234","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Evaluating L2 textbook input for grammar learning: from research findings to operationalized evaluation criteria 评价第二语言教材输入对语法学习的影响：从研究结果到可操作的评价标准

Research Methods in Applied Linguistics

Pub Date : 2025-06-09 DOI: 10.1016/j.rmal.2025.100213

Raphaël Perrin, Anita Thomas

Prior studies investigating the quality of linguistic input in L2 textbooks have mainly focused on lexical frequencies for vocabulary development. The present study explores how previous findings on the role of salience, frequency, and explicit instruction can be used in the form of operationalized criteria to evaluate input quality in L2 textbooks for the learning of grammatical target features. To this aim, we have reviewed research on the subject, suggested evaluation criteria based on this literature, and chosen two French textbooks and two sample target features – the position of French reflexive pronouns and their coreference with the subject – to explore the applicability of the criteria.

The example analysis suggests that the developed criteria are applicable and useful for evaluating L2 textbook input quality for the learning of grammatical features. However, the example analysis also highlights several challenges, such as interpreting frequency counts in the absence of benchmarks. More complete textbook analyses are needed to test the applicability of the suggested approach.

以往对二语教材中语言输入质量的研究主要集中在词汇发展的词汇频率上。本研究探讨了如何将先前关于显著性、频率和显性指令的作用的研究结果以可操作化标准的形式用于评估第二语言教科书中语法目标特征学习的输入质量。为此，我们回顾了这一主题的研究，提出了基于这些文献的评价标准，并选择了两本法语教科书和两个样本目标特征-法语反身代词的位置及其与主题的共指性-来探讨标准的适用性。实例分析表明，该标准可用于评价第二语言教材语法特征学习输入质量。然而，示例分析也强调了一些挑战，例如在没有基准测试的情况下解释频率计数。需要更完整的教科书分析来测试所建议方法的适用性。

{"title":"Evaluating L2 textbook input for grammar learning: from research findings to operationalized evaluation criteria","authors":"Raphaël Perrin, Anita Thomas","doi":"10.1016/j.rmal.2025.100213","DOIUrl":"10.1016/j.rmal.2025.100213","url":null,"abstract":"<div><div>Prior studies investigating the quality of linguistic input in L2 textbooks have mainly focused on lexical frequencies for vocabulary development. The present study explores how previous findings on the role of salience, frequency, and explicit instruction can be used in the form of operationalized criteria to evaluate input quality in L2 textbooks for the learning of grammatical target features. To this aim, we have reviewed research on the subject, suggested evaluation criteria based on this literature, and chosen two French textbooks and two sample target features – the position of French reflexive pronouns and their coreference with the subject – to explore the applicability of the criteria.</div><div>The example analysis suggests that the developed criteria are applicable and useful for evaluating L2 textbook input quality for the learning of grammatical features. However, the example analysis also highlights several challenges, such as interpreting frequency counts in the absence of benchmarks. More complete textbook analyses are needed to test the applicability of the suggested approach.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 3","pages":"Article 100213"},"PeriodicalIF":0.0,"publicationDate":"2025-06-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144242860","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Genre-based fine-tuning of large language models with self-organizing maps for automated writing evaluation 基于体裁的大型语言模型微调与自动写作评估的自组织地图

Research Methods in Applied Linguistics

Pub Date : 2025-06-07 DOI: 10.1016/j.rmal.2025.100219

Stephanie Link, Robert Redmon, Martin Hagan

Automated Writing Evaluation (AWE) systems have significantly advanced in providing feedback for academic essay writing. However, their predominant focus on sentence-level features highlights the need for a broader approach to AWE development. While genre-based AWE systems aim to address the socio-rhetorical complexities of writing for specific audiences and purposes, their availability remains limited. This scarcity is largely due to methodological constraints in developing robust feedback engines that effectively support discipline-specific writing needs. This article describes a new method for fine-tuning large-language models (LLM) and evaluating model performance, which we refer to as G-FiT Mapping (Genre-based FIne-Tuning with self-organizing maps). This method utilizes semi-automated annotation of genre-based functional-rhetorical units of text to efficiently fine-tune an LLM and then uses self-organizing maps to evaluate and improve network performance. The G-FiT Mapping method resulted in a new automated feedback engine for an intelligent tutoring system called Dissemity, for DISSeminating research with clarITY, that supports discipline-specific, scientific writers in writing for publication. We demonstrate use of G-Fit Mapping for establishing measurable improvements in network performance, offering implications for network interpretation, genre-based AWE, and AI-based learning systems development.

自动写作评估（AWE）系统在为学术论文写作提供反馈方面取得了重大进展。然而，他们对句子级功能的主要关注突出了对AWE开发的更广泛方法的需求。虽然基于体裁的AWE系统旨在解决特定受众和目的的社会修辞复杂性，但它们的可用性仍然有限。这种稀缺性很大程度上是由于在开发有效支持特定学科写作需求的健壮反馈引擎方面的方法限制。本文描述了一种微调大语言模型（LLM）和评估模型性能的新方法，我们将其称为G-FiT Mapping（基于类型的自组织映射的微调）。该方法利用基于体裁的文本功能修辞单元的半自动注释来有效地微调LLM，然后使用自组织映射来评估和改进网络性能。G-FiT Mapping方法为一个名为“传播”的智能辅导系统提供了一个新的自动反馈引擎，该系统用于传播具有清晰度的研究，支持特定学科的科学作者撰写出版物。我们展示了使用G-Fit Mapping来建立可衡量的网络性能改进，为网络解释、基于类型的AWE和基于人工智能的学习系统开发提供了启示。

{"title":"Genre-based fine-tuning of large language models with self-organizing maps for automated writing evaluation","authors":"Stephanie Link, Robert Redmon, Martin Hagan","doi":"10.1016/j.rmal.2025.100219","DOIUrl":"10.1016/j.rmal.2025.100219","url":null,"abstract":"<div><div>Automated Writing Evaluation (AWE) systems have significantly advanced in providing feedback for academic essay writing. However, their predominant focus on sentence-level features highlights the need for a broader approach to AWE development. While genre-based AWE systems aim to address the socio-rhetorical complexities of writing for specific audiences and purposes, their availability remains limited. This scarcity is largely due to methodological constraints in developing robust feedback engines that effectively support discipline-specific writing needs. This article describes a new method for fine-tuning large-language models (LLM) and evaluating model performance, which we refer to as G-FiT Mapping (Genre-based FIne-Tuning with self-organizing maps). This method utilizes semi-automated annotation of genre-based functional-rhetorical units of text to efficiently fine-tune an LLM and then uses self-organizing maps to evaluate and improve network performance. The G-FiT Mapping method resulted in a new automated feedback engine for an intelligent tutoring system called Dissemity, for DISSeminating research with clarITY, that supports discipline-specific, scientific writers in writing for publication. We demonstrate use of G-Fit Mapping for establishing measurable improvements in network performance, offering implications for network interpretation, genre-based AWE, and AI-based learning systems development.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 3","pages":"Article 100219"},"PeriodicalIF":0.0,"publicationDate":"2025-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144242859","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Advancing language education with ChatGPT: A path to cultivate 21st-century digital skills 用ChatGPT推进语言教育：培养21世纪数字技能的途径

Research Methods in Applied Linguistics

Pub Date : 2025-06-03 DOI: 10.1016/j.rmal.2025.100218

Amir Reza Rahimi , Ramin Teimouri

Artificial Intelligence (AI) has become increasingly integrated into education, human life, and work as we continue to enter the 21st century, resulting in an increased expectation that learners and humans should possess more skills, referred to as 21st-century digital skills. These skills have become essential for learning, working, and living with Artificial Intelligence and the latest generation of Information and Communication Technologies (ICTs). While recent studies have examined Artificial Intelligence, particularly ChatGPT, in terms of its ability to assist language learners in developing their language skills as well as their subskills, they have not explored its role in the development of their 21st-century digital skills. For this sake, in this study the researchers integrated ChatGPT into the language classroom procedure in three high schools in Tehran, where the language teacher tried to cultivate 21st-century digital skills with ChatGPT and led students to take advantage of its features to improve their 21st-century digital skills throughout the academic year of 2023–24 and then filled out the study survey. The results of the Partial Least Square Modelling Approach (PLS-SEM) showed that ChatGPT's personalization, interactivity, accuracy, and responsiveness, along with its anthropomorphism, significantly shaped language learners' 21st-century digital skills, including critical thinking digital skills, information evaluation skills, creative digital skills, and problem-solving skills. In addition, the study found a sign of digital self-authenticity, where language learners perceived that AI-assisted language learning enhanced their 21st-century digital skills more than previous language learning contexts, albeit at the expense of their digital collaboration skills. Therefore, the study broadened the existing literature by focusing on the 21st-century digital skills of language learners, going beyond their language skills and sub-skills. It suggests that ChatGPT has the potential to enhance and develop language learners' 21st-century digital skills. However, language teachers must devise metrics to encourage learners to engage in both collaborative and personalized language learning with ChatGPT, thereby fostering the development of all 21st-century digital skills.

随着我们进入21世纪，人工智能（AI）越来越多地融入到教育、人类生活和工作中，导致人们对学习者和人类应该拥有更多技能的期望越来越高，被称为21世纪数字技能。这些技能对于人工智能和最新一代信息通信技术（ict）的学习、工作和生活至关重要。虽然最近的研究已经研究了人工智能，特别是ChatGPT，在帮助语言学习者发展语言技能和子技能方面的能力，但他们并没有探索它在21世纪数字技能发展中的作用。为此，在本研究中，研究人员将ChatGPT整合到德黑兰三所高中的语言课堂程序中，在2023-24学年期间，语言教师试图用ChatGPT培养21世纪的数字技能，并引导学生利用其特点提高他们的21世纪数字技能，然后填写研究调查。偏最小二乘建模方法（PLS-SEM）的结果表明，ChatGPT的个性化、互动性、准确性和响应性，以及它的拟人化，显著地塑造了语言学习者的21世纪数字技能，包括批判性思维数字技能、信息评估技能、创造性数字技能和解决问题的技能。此外，该研究还发现了数字自我真实性的迹象，语言学习者认为人工智能辅助语言学习比以前的语言学习环境更能提高他们的21世纪数字技能，尽管以牺牲他们的数字协作技能为代价。因此，本研究拓宽了现有文献，将重点放在了语言学习者的21世纪数字技能上，超越了他们的语言技能和子技能。研究表明，ChatGPT有潜力增强和发展语言学习者的21世纪数字技能。然而，语言教师必须制定指标，鼓励学习者使用ChatGPT进行协作和个性化的语言学习，从而促进所有21世纪数字技能的发展。

{"title":"Advancing language education with ChatGPT: A path to cultivate 21st-century digital skills","authors":"Amir Reza Rahimi , Ramin Teimouri","doi":"10.1016/j.rmal.2025.100218","DOIUrl":"10.1016/j.rmal.2025.100218","url":null,"abstract":"<div><div>Artificial Intelligence (AI) has become increasingly integrated into education, human life, and work as we continue to enter the 21st century, resulting in an increased expectation that learners and humans should possess more skills, referred to as 21st-century digital skills. These skills have become essential for learning, working, and living with Artificial Intelligence and the latest generation of Information and Communication Technologies (ICTs). While recent studies have examined Artificial Intelligence, particularly ChatGPT, in terms of its ability to assist language learners in developing their language skills as well as their subskills, they have not explored its role in the development of their 21st-century digital skills. For this sake, in this study the researchers integrated ChatGPT into the language classroom procedure in three high schools in Tehran, where the language teacher tried to cultivate 21st-century digital skills with ChatGPT and led students to take advantage of its features to improve their 21st-century digital skills throughout the academic year of 2023–24 and then filled out the study survey. The results of the Partial Least Square Modelling Approach (PLS-SEM) showed that ChatGPT's personalization, interactivity, accuracy, and responsiveness, along with its anthropomorphism, significantly shaped language learners' 21st-century digital skills, including critical thinking digital skills, information evaluation skills, creative digital skills, and problem-solving skills. In addition, the study found a sign of digital self-authenticity, where language learners perceived that AI-assisted language learning enhanced their 21st-century digital skills more than previous language learning contexts, albeit at the expense of their digital collaboration skills. Therefore, the study broadened the existing literature by focusing on the 21st-century digital skills of language learners, going beyond their language skills and sub-skills. It suggests that ChatGPT has the potential to enhance and develop language learners' 21st-century digital skills. However, language teachers must devise metrics to encourage learners to engage in both collaborative and personalized language learning with ChatGPT, thereby fostering the development of all 21st-century digital skills.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100218"},"PeriodicalIF":0.0,"publicationDate":"2025-06-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144204939","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Ethics for researching language and education: What the discourse of professional guidelines reveals 研究语言与教育的伦理：专业指南话语的启示

Research Methods in Applied Linguistics

Pub Date : 2025-06-02 DOI: 10.1016/j.rmal.2025.100221

Theresa Austin , Rosa Alejandra Medina Riveros Ph.D

How do the guidelines provided by language and literacy professional organizations configure the ethical stances of their members, who are practitioners and researchers? As part of a larger project involving the lived experiences of teacher educators and researchers who work with multilingual populations, we present an initial critical discourse study of four professional organizations’ ethical codes of conduct. This article focuses on ethics as it materializes through contemporary textual analysis of public guidelines readily accessible to educational linguistics professionals during the 2021–2024 timeframe. It examines how ethical statements operate on the imagination of professionals desiring membership. We employ critical discourse analysis (CDA), where Van Dijk’s (1993) definition of discourse conceptualizes how language use creates power to dominate, resist, and build social hierarchies. We characterize each code as within deontic ethics by interpreting how the choice of modality guides members’ actions and values. During 2021–24 our corpus included documents detailing guidelines from AERA (2011), MLA (1992), LRA (2016), and AAAL (2017).

Our findings highlight the characteristics of these ethical codes and their differences. These documents configure an assemblage that affects members through assumed values and obligations. Furthermore, we identify unexamined ethical needs that arise in actual lived realities of researchers that require more fluid, contingent, and responsive ethics; hence, we propose a critically and dialogically engaged stance open to periodic yet continual reformulations.

语言和扫盲专业组织提供的指导方针如何配置其成员（从业者和研究人员）的道德立场？作为一个更大的项目的一部分，我们对四个专业组织的道德行为准则进行了初步的批判性话语研究，该项目涉及教师、教育工作者和与多语言人群打交道的研究人员的生活经验。本文重点关注伦理学，因为它通过2021-2024年期间教育语言学专业人员容易获得的公共指南的当代文本分析而具体化。它考察了道德声明如何对渴望成为会员的专业人士的想象力起作用。我们采用批判性话语分析（CDA），其中Van Dijk（1993）对话语的定义概念化了语言使用如何创造支配、抵抗和建立社会等级的权力。我们通过解释方式的选择如何指导成员的行为和价值观，将每个代码描述为道义伦理。在2021-24年期间，我们的语料库包括AERA（2011）、MLA（1992）、LRA（2016）和AAAL（2017）的详细指南文件。我们的研究结果突出了这些道德准则的特点及其差异。这些文档配置一个通过假定的值和义务影响成员的程序集。此外，我们确定了在研究人员的实际生活现实中出现的未经审查的伦理需求，这些需求需要更多的流动性，偶然性和响应性伦理；因此，我们提出了一种批判性和对话参与的立场，对定期但持续的重新表述开放。

{"title":"Ethics for researching language and education: What the discourse of professional guidelines reveals","authors":"Theresa Austin , Rosa Alejandra Medina Riveros Ph.D","doi":"10.1016/j.rmal.2025.100221","DOIUrl":"10.1016/j.rmal.2025.100221","url":null,"abstract":"<div><div>How do the guidelines provided by language and literacy professional organizations configure the ethical stances of their members, who are practitioners and researchers? As part of a larger project involving the lived experiences of teacher educators and researchers who work with multilingual populations, we present an initial critical discourse study of four professional organizations’ ethical codes of conduct. This article focuses on ethics as it materializes through contemporary textual analysis of public guidelines readily accessible to educational linguistics professionals during the 2021–2024 timeframe. It examines how ethical statements operate on the imagination of professionals desiring membership. We employ critical discourse analysis (CDA), where Van Dijk’s (1993) definition of discourse conceptualizes how language use creates power to dominate, resist, and build social hierarchies. We characterize each code as within deontic ethics by interpreting how the choice of modality guides members’ actions and values. During 2021–24 our corpus included documents detailing guidelines from AERA (2011), MLA (1992), LRA (2016), and AAAL (2017).</div><div>Our findings highlight the characteristics of these ethical codes and their differences. These documents configure an assemblage that affects members through assumed values and obligations. Furthermore, we identify unexamined ethical needs that arise in actual lived realities of researchers that require more fluid, contingent, and responsive ethics; hence, we propose a critically and dialogically engaged stance open to periodic yet continual reformulations.</div></div>","PeriodicalId":101075,"journal":{"name":"Research Methods in Applied Linguistics","volume":"4 2","pages":"Article 100221"},"PeriodicalIF":0.0,"publicationDate":"2025-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144196368","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0