“Significance sells”: Applied linguists’ views on questionable research practices

Abstract

Building on the increased concern over methodological quality and research judgment in applied linguistics and elsewhere in the social sciences (e.g., Gass et al., 2021; Open Science Collaboration, 2015), Isbell et al. (2022) examined the presence of questionable research practices (QRPs) as well as misconduct (e.g., data fabrication) in applied linguistics. The findings, based on a survey of 351 scholars, paint a disappointing view of the ethics of quantitative researchers in the field. Approximately 17% of the sample admitted to one or more forms of fraud, and nearly all (94%) reported engaging in one or more QRPs. Isbell et al. also collected responses from participants regarding their views of the different QRPs in the survey. The present study focuses on those responses in an attempt to shed light on the choices researchers make while handling, analyzing, and reporting quantitative data. Nine themes, such as researcher training and the changing landscape of applied linguistics research methods, were extracted from participants’ comments which were analyzed vis-à-vis aggregate-level survey data in Isbell et al. We also highlight the context-dependent nature of methodological choices. Representative responses are highlighted and discussed. Recommendations are also provided to guide researcher practice as well as for graduate training and field-wide standards.