Quantitative experimental L2 acquisition MALL studies: A critical evaluation of research quality

IF 4.6 1区 文学 Q1 EDUCATION & EDUCATIONAL RESEARCH Recall Pub Date : 2023-06-20 DOI:10.1017/s0958344023000149
Jack Burston, A. Athanasiou, K. Giannakou
{"title":"Quantitative experimental L2 acquisition MALL studies: A critical evaluation of research quality","authors":"Jack Burston, A. Athanasiou, K. Giannakou","doi":"10.1017/s0958344023000149","DOIUrl":null,"url":null,"abstract":"\n With more than 1,200 publications over the past two decades, experimental mobile-assisted language learning (MALL) studies targeting second/foreign language (L2) acquisition outcomes are certainly not lacking in quantity. Their research quality, on the other hand, has often been brought into question, most notably with regard to the adequacy of their assessment instruments and statistical analyses. Yet limiting the determination of research quality to the evaluation of testing procedures, and the statistical analysis of the results they produce, ignores the critical relevance of the underlying research parameters that generate the results in the first place. A comprehensive evaluation of quantitative experimental L2 acquisition MALL research quality, encompassing design as well as assessment instruments and statistical analysis, thus remains to be undertaken. The present investigation endeavors to do so based on an extensive compilation of 737 MALL studies published between 2000 and 2021. The research quality of these publications is evaluated according to four main parameters: language acquisition moderators, treatment intervention conditions, assessment instruments, and statistical analysis. These are applied according to a modified version of the Checklist for the Rigor of Education-Experiment Designs (CREED), which classifies research design quality into five levels: low, medium-low, medium, medium-high, high. With over three quarters of all studies falling within the low category, the result leaves much to be desired. Since the modified CREED algorithm developed here can equally be applied to studies from their inception, it offers a way forward to improve the research quality of future experimental MALL studies.","PeriodicalId":47046,"journal":{"name":"Recall","volume":"1 1","pages":""},"PeriodicalIF":4.6000,"publicationDate":"2023-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Recall","FirstCategoryId":"98","ListUrlMain":"https://doi.org/10.1017/s0958344023000149","RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0

Abstract

With more than 1,200 publications over the past two decades, experimental mobile-assisted language learning (MALL) studies targeting second/foreign language (L2) acquisition outcomes are certainly not lacking in quantity. Their research quality, on the other hand, has often been brought into question, most notably with regard to the adequacy of their assessment instruments and statistical analyses. Yet limiting the determination of research quality to the evaluation of testing procedures, and the statistical analysis of the results they produce, ignores the critical relevance of the underlying research parameters that generate the results in the first place. A comprehensive evaluation of quantitative experimental L2 acquisition MALL research quality, encompassing design as well as assessment instruments and statistical analysis, thus remains to be undertaken. The present investigation endeavors to do so based on an extensive compilation of 737 MALL studies published between 2000 and 2021. The research quality of these publications is evaluated according to four main parameters: language acquisition moderators, treatment intervention conditions, assessment instruments, and statistical analysis. These are applied according to a modified version of the Checklist for the Rigor of Education-Experiment Designs (CREED), which classifies research design quality into five levels: low, medium-low, medium, medium-high, high. With over three quarters of all studies falling within the low category, the result leaves much to be desired. Since the modified CREED algorithm developed here can equally be applied to studies from their inception, it offers a way forward to improve the research quality of future experimental MALL studies.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
定量实验二语习得MALL研究:研究质量的关键评估
在过去的二十年里,有1200多篇出版物,针对第二语言/外语(L2)习得结果的实验性移动辅助语言学习(MALL)研究在数量上肯定并不缺乏。另一方面,他们的研究质量经常受到质疑,尤其是在评估工具和统计分析的充分性方面。然而,将研究质量的确定局限于测试程序的评估,以及对其产生的结果的统计分析,忽略了最初产生结果的基本研究参数的关键相关性。因此,对定量实验L2习得MALL研究质量的全面评估,包括设计、评估工具和统计分析,仍有待进行。本调查基于2000年至2021年间发表的737项MALL研究的广泛汇编。这些出版物的研究质量根据四个主要参数进行评估:语言习得调节因子、治疗干预条件、评估工具和统计分析。这些是根据《教育实验设计严格性检查表》(CREED)的修订版应用的,该检查表将研究设计质量分为五个级别:低、中低、中、中高、高。由于超过四分之三的研究属于低类,结果还有很多不足之处。由于这里开发的改进的CREED算法从一开始就可以同样应用于研究,它为提高未来实验MALL研究的研究质量提供了一条前进的道路。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Recall
Recall Multiple-
CiteScore
8.50
自引率
4.40%
发文量
17
期刊最新文献
Forty-two years of computer-assisted language learning research: A scientometric study of hotspot research and trending issues Different interlocutors, different EFL interactional strategies: A case study of intercultural telecollaborative projects in secondary classrooms Examining the relationships among motivation, informal digital learning of English, and foreign language enjoyment: An explanatory mixed-method study ReCALL editorial September 2023 issue Sampling and randomisation in experimental and quasi-experimental CALL studies: Issues and recommendations for design, reporting, review, and interpretation
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1