随机试验中统计结果的解释:利用开放式问题的主题分析对统计人员进行的调查。

IF 3.9 3区 医学 Q1 HEALTH CARE SCIENCES & SERVICES BMC Medical Research Methodology Pub Date : 2024-10-29 DOI:10.1186/s12874-024-02366-4
Karla Hemming, Laura Kudrna, Sam Watson, Monica Taljaard, Sheila Greenfield, Beatriz Goulao, Richard Lilford
{"title":"随机试验中统计结果的解释:利用开放式问题的主题分析对统计人员进行的调查。","authors":"Karla Hemming, Laura Kudrna, Sam Watson, Monica Taljaard, Sheila Greenfield, Beatriz Goulao, Richard Lilford","doi":"10.1186/s12874-024-02366-4","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Dichotomisation of statistical significance, rather than interpretation of effect sizes supported by confidence intervals, is a long-standing problem.</p><p><strong>Methods: </strong>We distributed an online survey to clinical trial statisticians across the UK, Australia and Canada asking about their experiences, perspectives and practices with respect to interpretation of statistical findings from randomised trials. We report a descriptive analysis of the closed-ended questions and a thematic analysis of the open-ended questions.</p><p><strong>Results: </strong>We obtained 101 responses across a broad range of career stages (24% professors; 51% senior lecturers; 22% junior statisticians) and areas of work (28% early phase trials; 44% drug trials; 38% health service trials). The majority (93%) believed that statistical findings should be interpreted by considering (minimal) clinical importance of treatment effects, but many (61%) said quantifying clinically important effect sizes was difficult, and fewer (54%) followed this approach in practice. Thematic analysis identified several barriers to forming a consensus on the statistical interpretation of the study findings, including: the dynamics within teams, lack of knowledge or difficulties in communicating that knowledge, as well as external pressures. External pressures included the pressure to publish definitive findings and statistical review which can sometimes be unhelpful but can at times be a saving grace. However, the concept of the minimally important difference was identified as a particularly poorly defined, even nebulous, construct which lies at the heart of much disagreement and confusion in the field.</p><p><strong>Conclusion: </strong>The majority of participating statisticians believed that it is important to interpret statistical findings based on the clinically important effect size, but report this is difficult to operationalise. Reaching a consensus on the interpretation of a study is a social process involving disparate members of the research team along with editors and reviewers, as well as patients who likely have a role in the elicitation of minimally important differences.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":null,"pages":null},"PeriodicalIF":3.9000,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11520448/pdf/","citationCount":"0","resultStr":"{\"title\":\"Interpretation of statistical findings in randomised trials: a survey of statisticians using thematic analysis of open-ended questions.\",\"authors\":\"Karla Hemming, Laura Kudrna, Sam Watson, Monica Taljaard, Sheila Greenfield, Beatriz Goulao, Richard Lilford\",\"doi\":\"10.1186/s12874-024-02366-4\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Background: </strong>Dichotomisation of statistical significance, rather than interpretation of effect sizes supported by confidence intervals, is a long-standing problem.</p><p><strong>Methods: </strong>We distributed an online survey to clinical trial statisticians across the UK, Australia and Canada asking about their experiences, perspectives and practices with respect to interpretation of statistical findings from randomised trials. We report a descriptive analysis of the closed-ended questions and a thematic analysis of the open-ended questions.</p><p><strong>Results: </strong>We obtained 101 responses across a broad range of career stages (24% professors; 51% senior lecturers; 22% junior statisticians) and areas of work (28% early phase trials; 44% drug trials; 38% health service trials). The majority (93%) believed that statistical findings should be interpreted by considering (minimal) clinical importance of treatment effects, but many (61%) said quantifying clinically important effect sizes was difficult, and fewer (54%) followed this approach in practice. Thematic analysis identified several barriers to forming a consensus on the statistical interpretation of the study findings, including: the dynamics within teams, lack of knowledge or difficulties in communicating that knowledge, as well as external pressures. External pressures included the pressure to publish definitive findings and statistical review which can sometimes be unhelpful but can at times be a saving grace. However, the concept of the minimally important difference was identified as a particularly poorly defined, even nebulous, construct which lies at the heart of much disagreement and confusion in the field.</p><p><strong>Conclusion: </strong>The majority of participating statisticians believed that it is important to interpret statistical findings based on the clinically important effect size, but report this is difficult to operationalise. Reaching a consensus on the interpretation of a study is a social process involving disparate members of the research team along with editors and reviewers, as well as patients who likely have a role in the elicitation of minimally important differences.</p>\",\"PeriodicalId\":9114,\"journal\":{\"name\":\"BMC Medical Research Methodology\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":3.9000,\"publicationDate\":\"2024-10-29\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11520448/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"BMC Medical Research Methodology\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1186/s12874-024-02366-4\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"HEALTH CARE SCIENCES & SERVICES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"BMC Medical Research Methodology","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1186/s12874-024-02366-4","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
引用次数: 0

摘要

背景:统计显著性的二分法,而非置信区间支持的效应大小解释,是一个长期存在的问题:统计显著性的二分法,而不是在置信区间支持下解释效应大小,是一个长期存在的问题:我们向英国、澳大利亚和加拿大的临床试验统计人员发放了一份在线调查问卷,询问他们在解释随机试验统计结果方面的经验、观点和做法。我们对封闭式问题进行了描述性分析,对开放式问题进行了主题分析:我们获得了 101 份答复,这些答复涉及不同的职业阶段(24% 教授;51% 高级讲师;22% 初级统计员)和工作领域(28% 早期试验;44% 药物试验;38% 医疗服务试验)。大多数人(93%)认为,应通过考虑治疗效果的(最小)临床重要性来解释统计结果,但许多人(61%)表示,量化具有临床重要性的效应大小很困难,而在实践中采用这种方法的人数较少(54%)。专题分析发现了就研究结果的统计学解释达成共识的几个障碍,包括:团队内部的动态、缺乏知识或难以传达知识,以及外部压力。外部压力包括发表最终研究结果和统计审查的压力,这有时可能无益,但有时也是一种拯救。然而,"最小重要差异 "的概念被认为是一个定义不清、甚至模糊不清的概念,是该领域中许多分歧和混乱的核心所在:大多数参与研究的统计学家认为,根据临床重要效应大小来解释统计结果非常重要,但报告称这很难操作化。就一项研究的解释达成共识是一个社会过程,涉及到研究团队的不同成员、编辑和审稿人,以及可能在最小重要差异的激发中发挥作用的患者。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Interpretation of statistical findings in randomised trials: a survey of statisticians using thematic analysis of open-ended questions.

Background: Dichotomisation of statistical significance, rather than interpretation of effect sizes supported by confidence intervals, is a long-standing problem.

Methods: We distributed an online survey to clinical trial statisticians across the UK, Australia and Canada asking about their experiences, perspectives and practices with respect to interpretation of statistical findings from randomised trials. We report a descriptive analysis of the closed-ended questions and a thematic analysis of the open-ended questions.

Results: We obtained 101 responses across a broad range of career stages (24% professors; 51% senior lecturers; 22% junior statisticians) and areas of work (28% early phase trials; 44% drug trials; 38% health service trials). The majority (93%) believed that statistical findings should be interpreted by considering (minimal) clinical importance of treatment effects, but many (61%) said quantifying clinically important effect sizes was difficult, and fewer (54%) followed this approach in practice. Thematic analysis identified several barriers to forming a consensus on the statistical interpretation of the study findings, including: the dynamics within teams, lack of knowledge or difficulties in communicating that knowledge, as well as external pressures. External pressures included the pressure to publish definitive findings and statistical review which can sometimes be unhelpful but can at times be a saving grace. However, the concept of the minimally important difference was identified as a particularly poorly defined, even nebulous, construct which lies at the heart of much disagreement and confusion in the field.

Conclusion: The majority of participating statisticians believed that it is important to interpret statistical findings based on the clinically important effect size, but report this is difficult to operationalise. Reaching a consensus on the interpretation of a study is a social process involving disparate members of the research team along with editors and reviewers, as well as patients who likely have a role in the elicitation of minimally important differences.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
BMC Medical Research Methodology
BMC Medical Research Methodology 医学-卫生保健
CiteScore
6.50
自引率
2.50%
发文量
298
审稿时长
3-8 weeks
期刊介绍: BMC Medical Research Methodology is an open access journal publishing original peer-reviewed research articles in methodological approaches to healthcare research. Articles on the methodology of epidemiological research, clinical trials and meta-analysis/systematic review are particularly encouraged, as are empirical studies of the associations between choice of methodology and study outcomes. BMC Medical Research Methodology does not aim to publish articles describing scientific methods or techniques: these should be directed to the BMC journal covering the relevant biomedical subject area.
期刊最新文献
Motivations for enrollment in a COVID-19 ring-based post-exposure prophylaxis trial: qualitative examination of participant experiences. Concordance between humans and GPT-4 in appraising the methodological quality of case reports and case series using the Murad tool. Bayesian additive regression trees for predicting childhood asthma in the CHILD cohort study. Incorporating external controls in the design of randomized clinical trials: a case study in solid tumors. Recruiting and retaining healthcare workers in Scotland to a longitudinal COVID-19 study: a descriptive analysis.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1