The policies on the use of large language models in radiological journals are lacking: a meta-research study.

IF 4.1 2区医学 Q1 RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING Insights into Imaging Pub Date : 2024-08-01 DOI:10.1186/s13244-024-01769-7

Jingyu Zhong, Yue Xing, Yangfan Hu, Junjie Lu, Jiarui Yang, Guangcheng Zhang, Shiqi Mao, Haoda Chen, Qian Yin, Qingqing Cen, Run Jiang, Jingshen Chu, Yang Song, Minda Lu, Defang Ding, Xiang Ge, Huan Zhang, Weiwu Yao

{"title":"The policies on the use of large language models in radiological journals are lacking: a meta-research study.","authors":"Jingyu Zhong, Yue Xing, Yangfan Hu, Junjie Lu, Jiarui Yang, Guangcheng Zhang, Shiqi Mao, Haoda Chen, Qian Yin, Qingqing Cen, Run Jiang, Jingshen Chu, Yang Song, Minda Lu, Defang Ding, Xiang Ge, Huan Zhang, Weiwu Yao","doi":"10.1186/s13244-024-01769-7","DOIUrl":null,"url":null,"abstract":"Objective: To evaluate whether and how the radiological journals present their policies on the use of large language models (LLMs), and identify the journal characteristic variables that are associated with the presence.Methods: In this meta-research study, we screened Journals from the Radiology, Nuclear Medicine and Medical Imaging Category, 2022 Journal Citation Reports, excluding journals in non-English languages and relevant documents unavailable. We assessed their LLM use policies: (1) whether the policy is present; (2) whether the policy for the authors, the reviewers, and the editors is present; and (3) whether the policy asks the author to report the usage of LLMs, the name of LLMs, the section that used LLMs, the role of LLMs, the verification of LLMs, and the potential influence of LLMs. The association between the presence of policies and journal characteristic variables was evaluated.Results: The LLM use policies were presented in 43.9% (83/189) of journals, and those for the authors, the reviewers, and the editor were presented in 43.4% (82/189), 29.6% (56/189) and 25.9% (49/189) of journals, respectively. Many journals mentioned the aspects of the usage (43.4%, 82/189), the name (34.9%, 66/189), the verification (33.3%, 63/189), and the role (31.7%, 60/189) of LLMs, while the potential influence of LLMs (4.2%, 8/189), and the section that used LLMs (1.6%, 3/189) were seldomly touched. The publisher is related to the presence of LLM use policies (p < 0.001).Conclusion: The presence of LLM use policies is suboptimal in radiological journals. A reporting guideline is encouraged to facilitate reporting quality and transparency.Critical relevance statement: It may facilitate the quality and transparency of the use of LLMs in scientific writing if a shared complete reporting guideline is developed by stakeholders and then endorsed by journals.Key points: The policies on LLM use in radiological journals are unexplored. Some of the radiological journals presented policies on LLM use. A shared complete reporting guideline for LLM use is desired.","PeriodicalId":13639,"journal":{"name":"Insights into Imaging","volume":"15 1","pages":"186"},"PeriodicalIF":4.1000,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11294318/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Insights into Imaging","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1186/s13244-024-01769-7","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING","Score":null,"Total":0}

引用次数: 0

Abstract

Objective: To evaluate whether and how the radiological journals present their policies on the use of large language models (LLMs), and identify the journal characteristic variables that are associated with the presence.

Methods: In this meta-research study, we screened Journals from the Radiology, Nuclear Medicine and Medical Imaging Category, 2022 Journal Citation Reports, excluding journals in non-English languages and relevant documents unavailable. We assessed their LLM use policies: (1) whether the policy is present; (2) whether the policy for the authors, the reviewers, and the editors is present; and (3) whether the policy asks the author to report the usage of LLMs, the name of LLMs, the section that used LLMs, the role of LLMs, the verification of LLMs, and the potential influence of LLMs. The association between the presence of policies and journal characteristic variables was evaluated.

Results: The LLM use policies were presented in 43.9% (83/189) of journals, and those for the authors, the reviewers, and the editor were presented in 43.4% (82/189), 29.6% (56/189) and 25.9% (49/189) of journals, respectively. Many journals mentioned the aspects of the usage (43.4%, 82/189), the name (34.9%, 66/189), the verification (33.3%, 63/189), and the role (31.7%, 60/189) of LLMs, while the potential influence of LLMs (4.2%, 8/189), and the section that used LLMs (1.6%, 3/189) were seldomly touched. The publisher is related to the presence of LLM use policies (p < 0.001).

Conclusion: The presence of LLM use policies is suboptimal in radiological journals. A reporting guideline is encouraged to facilitate reporting quality and transparency.

Critical relevance statement: It may facilitate the quality and transparency of the use of LLMs in scientific writing if a shared complete reporting guideline is developed by stakeholders and then endorsed by journals.

Key points: The policies on LLM use in radiological journals are unexplored. Some of the radiological journals presented policies on LLM use. A shared complete reporting guideline for LLM use is desired.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

放射学期刊缺乏使用大型语言模型的政策：一项元研究。

目的评估放射学期刊是否以及如何介绍其使用大语言模型（LLM）的政策，并确定与是否存在大语言模型相关的期刊特征变量：在这项荟萃研究中，我们筛选了放射学、核医学和医学影像类期刊、2022 期刊引文报告，排除了非英语期刊和无法获得的相关文献。我们评估了它们的LLM使用政策：（1）是否存在该政策；（2）是否存在针对作者、审稿人和编辑的政策；（3）该政策是否要求作者报告LLM的使用情况、LLM的名称、使用LLM的章节、LLM的作用、LLM的验证以及LLM的潜在影响。评估了政策的存在与期刊特征变量之间的关联：43.9%的期刊（83/189）介绍了法律硕士使用政策，43.4%的期刊（82/189）、29.6%的期刊（56/189）和25.9%的期刊（49/189）分别介绍了针对作者、审稿人和编辑的政策。许多期刊提到了法学硕士的用途（43.4%，82/189）、名称（34.9%，66/189）、验证（33.3%，63/189）和作用（31.7%，60/189），而很少涉及法学硕士的潜在影响（4.2%，8/189）和使用法学硕士的栏目（1.6%，3/189）。出版商与是否存在使用当地语言学习者的政策有关（p 结论）：在放射学期刊中，LLM 使用政策的存在情况并不理想。鼓励制定报告指南，以提高报告质量和透明度：如果利益相关者共同制定一份完整的报告指南，然后得到期刊的认可，将有助于提高科学写作中使用LLM的质量和透明度：放射学期刊使用LLM的政策尚未得到探讨。一些放射学期刊提出了使用LLM的政策。希望制定一份关于LLM使用的共享完整报告指南。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Insights into Imaging Medicine-Radiology, Nuclear Medicine and Imaging

CiteScore

7.30

自引率

4.30%

发文量

182

审稿时长

13 weeks

期刊介绍： Insights into Imaging (I³) is a peer-reviewed open access journal published under the brand SpringerOpen. All content published in the journal is freely available online to anyone, anywhere! I³ continuously updates scientific knowledge and progress in best-practice standards in radiology through the publication of original articles and state-of-the-art reviews and opinions, along with recommendations and statements from the leading radiological societies in Europe. Founded by the European Society of Radiology (ESR), I³ creates a platform for educational material, guidelines and recommendations, and a forum for topics of controversy. A balanced combination of review articles, original papers, short communications from European radiological congresses and information on society matters makes I³ an indispensable source for current information in this field. I³ is owned by the ESR, however authors retain copyright to their article according to the Creative Commons Attribution License (see Copyright and License Agreement). All articles can be read, redistributed and reused for free, as long as the author of the original work is cited properly. The open access fees (article-processing charges) for this journal are kindly sponsored by ESR for all Members. The journal went open access in 2012, which means that all articles published since then are freely available online.