首页 > 最新文献

BMC Medical Research Methodology最新文献

英文 中文
Framework for types of metainferences in mixed methods research.
IF 3.9 3区 医学 Q1 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-01-24 DOI: 10.1186/s12874-025-02475-8
Ahtisham Younas, Sergi Fàbregues, Sarah Munce, John W Creswell

Background: The generation of metainferences is a core and significant feature of mixed methods research. In recent years, there has been some discussion in the literature about criteria for appraising the quality of metainferences, the processes for generating them, and the critical role that assessing the "fit" of quantitative and qualitative data and results plays in this generative process. However, little is known about the types of insights that emerge from generating metainferences. To address this gap, this paper conceptualize and present the types and forms of metainferences that can be generated in MMR studies for guiding future research projects.

Methods: A critical review of literature sources was conducted, including peer-reviewed articles, book chapters, and research reports. We performed a non-systematic literature search in the Scopus, Web of Science, Ovid, and Google Scholar databases using general phrases such as "inferences in research", "metainferences in mixed methods", "inferences in mixed methods research", and "inference types". Additional searches included key methodological journals, such as the Journal of Mixed Methods Research, International Journal of Multiple Research Approaches, Methodological Innovations, and the Sage Research Methods database, to locate books, chapters, and peer-reviewed articles that discussed inferences and metainferences.

Results: We propose two broad types of metainferences and five sub-types. The broad metainferences are global and specific, and the subtypes include relational, predictive, causal, comparative, and elaborative metainferences. Furthermore, we provide examples of each type of metainference from published mixed methods empirical studies.

Conclusions: This paper contributes to the field of mixed methods research by expanding the knowledge about metainferences and offering a practical framework of types of metainferences for mixed methods researchers and educators. The proposed framework offers an approach to identifying and recognizing types of metainferences in mixed methods research and serves as an opportunity for future discussion on the nature, insights, and characteristic features of metainferences within this methodology. By proposing a foundation for metainferences, our framework advances this critical area of mixed methods research.

{"title":"Framework for types of metainferences in mixed methods research.","authors":"Ahtisham Younas, Sergi Fàbregues, Sarah Munce, John W Creswell","doi":"10.1186/s12874-025-02475-8","DOIUrl":"10.1186/s12874-025-02475-8","url":null,"abstract":"<p><strong>Background: </strong>The generation of metainferences is a core and significant feature of mixed methods research. In recent years, there has been some discussion in the literature about criteria for appraising the quality of metainferences, the processes for generating them, and the critical role that assessing the \"fit\" of quantitative and qualitative data and results plays in this generative process. However, little is known about the types of insights that emerge from generating metainferences. To address this gap, this paper conceptualize and present the types and forms of metainferences that can be generated in MMR studies for guiding future research projects.</p><p><strong>Methods: </strong>A critical review of literature sources was conducted, including peer-reviewed articles, book chapters, and research reports. We performed a non-systematic literature search in the Scopus, Web of Science, Ovid, and Google Scholar databases using general phrases such as \"inferences in research\", \"metainferences in mixed methods\", \"inferences in mixed methods research\", and \"inference types\". Additional searches included key methodological journals, such as the Journal of Mixed Methods Research, International Journal of Multiple Research Approaches, Methodological Innovations, and the Sage Research Methods database, to locate books, chapters, and peer-reviewed articles that discussed inferences and metainferences.</p><p><strong>Results: </strong>We propose two broad types of metainferences and five sub-types. The broad metainferences are global and specific, and the subtypes include relational, predictive, causal, comparative, and elaborative metainferences. Furthermore, we provide examples of each type of metainference from published mixed methods empirical studies.</p><p><strong>Conclusions: </strong>This paper contributes to the field of mixed methods research by expanding the knowledge about metainferences and offering a practical framework of types of metainferences for mixed methods researchers and educators. The proposed framework offers an approach to identifying and recognizing types of metainferences in mixed methods research and serves as an opportunity for future discussion on the nature, insights, and characteristic features of metainferences within this methodology. By proposing a foundation for metainferences, our framework advances this critical area of mixed methods research.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"18"},"PeriodicalIF":3.9,"publicationDate":"2025-01-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11758751/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143036861","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Analysis methods for covariate-constrained cluster randomized trials with time-to-event outcomes.
IF 3.9 3区 医学 Q1 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-01-22 DOI: 10.1186/s12874-025-02465-w
Amy M Crisp, M Elizabeth Halloran, Matt D T Hitchings, Ira M Longini, Natalie E Dean

Background: Cluster randomized trials, which often enroll a small number of clusters, can benefit from constrained randomization, selecting a final randomization scheme from a set of known, balanced randomizations. Previous literature has addressed the suitability of adjusting the analysis for the covariates that were balanced in the design phase when the outcome is continuous or binary. Here we extended this work to time-to-event outcomes by comparing two model-based tests and a newly derived permutation test. A current cluster randomized trial of vector control for the prevention of mosquito-borne disease in children in Mexico is used as a motivating example.

Methods: We assessed type I error rates and power between simple randomization and constrained randomization using both prognostic and non-prognostic covariates via a simulation study. We compared the performance of a semi-parametric Cox proportional hazards model with robust variance, a mixed effects Cox model, and a permutation test utilizing deviance residuals.

Results: The permutation test generally maintained nominal type I error-with the exception of the unadjusted analysis for constrained randomization-and also provided power comparable to the two Cox model-based tests. The model-based tests had inflated type I error when there were very few clusters per trial arm. All three methods performed well when there were 25 clusters per trial arm, as in the case of the motivating example.

Conclusion: For time-to-event outcomes, covariate-constrained randomization was shown to improve power relative to simple randomization. The permutation test developed here was more robust to inflation of type I error compared to model-based tests. Gaining power by adjusting for covariates in the analysis phase was largely dependent on the number of clusters per trial arm.

{"title":"Analysis methods for covariate-constrained cluster randomized trials with time-to-event outcomes.","authors":"Amy M Crisp, M Elizabeth Halloran, Matt D T Hitchings, Ira M Longini, Natalie E Dean","doi":"10.1186/s12874-025-02465-w","DOIUrl":"10.1186/s12874-025-02465-w","url":null,"abstract":"<p><strong>Background: </strong>Cluster randomized trials, which often enroll a small number of clusters, can benefit from constrained randomization, selecting a final randomization scheme from a set of known, balanced randomizations. Previous literature has addressed the suitability of adjusting the analysis for the covariates that were balanced in the design phase when the outcome is continuous or binary. Here we extended this work to time-to-event outcomes by comparing two model-based tests and a newly derived permutation test. A current cluster randomized trial of vector control for the prevention of mosquito-borne disease in children in Mexico is used as a motivating example.</p><p><strong>Methods: </strong>We assessed type I error rates and power between simple randomization and constrained randomization using both prognostic and non-prognostic covariates via a simulation study. We compared the performance of a semi-parametric Cox proportional hazards model with robust variance, a mixed effects Cox model, and a permutation test utilizing deviance residuals.</p><p><strong>Results: </strong>The permutation test generally maintained nominal type I error-with the exception of the unadjusted analysis for constrained randomization-and also provided power comparable to the two Cox model-based tests. The model-based tests had inflated type I error when there were very few clusters per trial arm. All three methods performed well when there were 25 clusters per trial arm, as in the case of the motivating example.</p><p><strong>Conclusion: </strong>For time-to-event outcomes, covariate-constrained randomization was shown to improve power relative to simple randomization. The permutation test developed here was more robust to inflation of type I error compared to model-based tests. Gaining power by adjusting for covariates in the analysis phase was largely dependent on the number of clusters per trial arm.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"16"},"PeriodicalIF":3.9,"publicationDate":"2025-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11753003/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143022122","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
The proper application of logistic regression model in complex survey data: a systematic review.
IF 3.9 3区 医学 Q1 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-01-22 DOI: 10.1186/s12874-024-02454-5
Devjit Dey, Md Samio Haque, Md Mojahedul Islam, Umme Iffat Aishi, Sajida Sultana Shammy, Md Sabbir Ahmed Mayen, Syed Toukir Ahmed Noor, Md Jamal Uddin

Background: Logistic regression is a useful statistical technique commonly used in many fields like healthcare, marketing, or finance to generate insights from binary outcomes (e.g., sick vs. not sick). However, when applying logistic regression to complex survey data, which includes complex sampling designs, specific methodological issues are often overlooked.

Methods: The systematic review extensively searched the PubMed and ScienceDirect databases from January 2015 to December 2021, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines, focusing primarily on the Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS). 810 articles met the inclusion criteria and were included in the analysis. When discussing logistic regression, the review considered multiple methodological problems such as the model adequacy assessment, handling dependence of observations, utilization of complex survey design, dealing with missing values, outliers, and more.

Results: Among the selected articles, the DHS database was used the most (96%), with MICS accounting for only 3%, and both DHS and MICS accounting for 1%. Of these, it was found that only 19.7% of the studies employed multilevel mixed-effects logistic regression to account for data dependencies. Model validation techniques were not reported in 94.8% of the studies with limited uses of the bootstrap, jackknife, and other resampling methods. Moreover, sample weights, PSUs, and strata variables were used together in 40.4% of the articles, and 41.7% of the studies did not use any of these variables, which could have produced biased results. Goodness-of-fit assessments were not mentioned in 75.3% of the articles, and the Hosmer-Lemeshow and likelihood ratio test were the most common among those reported. Furthermore, 95.8% of studies did not mention outliers, and only 41.0% of studies corrected for missing information, while only 2.7% applied imputation techniques.

Conclusions: This systematic review highlights important gaps in the use of logistic regression with complex survey data, such as overlooking data dependencies, survey design, and proper validation techniques, along with neglecting outliers, missing data, and goodness-of-fit assessments, all of which point to the need for clearer methodological standards and more thorough reporting to improve the reliability of results. Future research should focus on consistently following these standards to ensure stronger and more dependable findings.

{"title":"The proper application of logistic regression model in complex survey data: a systematic review.","authors":"Devjit Dey, Md Samio Haque, Md Mojahedul Islam, Umme Iffat Aishi, Sajida Sultana Shammy, Md Sabbir Ahmed Mayen, Syed Toukir Ahmed Noor, Md Jamal Uddin","doi":"10.1186/s12874-024-02454-5","DOIUrl":"10.1186/s12874-024-02454-5","url":null,"abstract":"<p><strong>Background: </strong>Logistic regression is a useful statistical technique commonly used in many fields like healthcare, marketing, or finance to generate insights from binary outcomes (e.g., sick vs. not sick). However, when applying logistic regression to complex survey data, which includes complex sampling designs, specific methodological issues are often overlooked.</p><p><strong>Methods: </strong>The systematic review extensively searched the PubMed and ScienceDirect databases from January 2015 to December 2021, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines, focusing primarily on the Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS). 810 articles met the inclusion criteria and were included in the analysis. When discussing logistic regression, the review considered multiple methodological problems such as the model adequacy assessment, handling dependence of observations, utilization of complex survey design, dealing with missing values, outliers, and more.</p><p><strong>Results: </strong>Among the selected articles, the DHS database was used the most (96%), with MICS accounting for only 3%, and both DHS and MICS accounting for 1%. Of these, it was found that only 19.7% of the studies employed multilevel mixed-effects logistic regression to account for data dependencies. Model validation techniques were not reported in 94.8% of the studies with limited uses of the bootstrap, jackknife, and other resampling methods. Moreover, sample weights, PSUs, and strata variables were used together in 40.4% of the articles, and 41.7% of the studies did not use any of these variables, which could have produced biased results. Goodness-of-fit assessments were not mentioned in 75.3% of the articles, and the Hosmer-Lemeshow and likelihood ratio test were the most common among those reported. Furthermore, 95.8% of studies did not mention outliers, and only 41.0% of studies corrected for missing information, while only 2.7% applied imputation techniques.</p><p><strong>Conclusions: </strong>This systematic review highlights important gaps in the use of logistic regression with complex survey data, such as overlooking data dependencies, survey design, and proper validation techniques, along with neglecting outliers, missing data, and goodness-of-fit assessments, all of which point to the need for clearer methodological standards and more thorough reporting to improve the reliability of results. Future research should focus on consistently following these standards to ensure stronger and more dependable findings.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"15"},"PeriodicalIF":3.9,"publicationDate":"2025-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11752662/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143022127","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Operationalising routinely collected patient data in research to further the pursuit of social justice and health equity: a team-based scoping review. 将常规收集的患者数据用于研究,以进一步追求社会公正和卫生公平:以团队为基础的范围审查。
IF 3.9 3区 医学 Q1 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-01-21 DOI: 10.1186/s12874-025-02466-9
Katie Chadd, Anna Caute, Anna Pettican, Pam Enderby

Background: Vast volumes of routinely collected data (RCD) about patients are collated by health professionals. Leveraging this data - a form of real-world data - can be valuable for quality improvement and contributing to the evidence-base to inform practice. Examining routine data may be especially useful for examining issues related to social justice such as health inequities. However, little is known about the extent to which RCD is utilised in health fields and published for wider dissemination.

Objectives: The objective of this scoping review is to document the peer-reviewed published research in allied health fields which utilise RCD and evaluate the extent to which these studies have addressed issues pertaining to social justice.

Methods: An enhanced version of the Arksey and O'Malley's framework, put forth by Westphalm et al. guided the scoping review. A comprehensive literature search of three databases identified 1584 articles. Application of inclusion and exclusion criteria was piloted on 5% of the papers by three researchers. All titles and abstracts were screened independently by 2 team members, as were full texts. A data charting framework, developed to address the research questions, was piloted by three researchers with data extraction being completed by the lead researcher. A sample of papers were independently charted by a second researcher for reliability checking.

Results: One hundred and ninety papers were included in the review. The literature was diverse in terms of the professions that were represented: physiotherapy (33.7%) and psychology/mental health professions (15.8%) predominated. Many studies were first authored by clinicians (44.2%), often with clinical-academic teams. Some (33.25%) directly referenced the use of their studies to examine translation of research to practice. Few studies (14.2%) specifically tackled issues pertaining to social justice, though many collected variables that could have been utilised for this purpose.

Conclusion: Studies operationalising RCD can meaningfully address research to practice gaps and provide new evidence about issues related to social justice. However, RCD is underutilised for these purposes. Given that vast volumes of relevant data are routinely collected, more needs to be done to leverage it, which would be supported by greater acknowledgement of the value of RCD studies.

背景:卫生专业人员对患者的大量常规收集数据(RCD)进行了整理。利用这些数据——一种真实世界数据的形式——对提高质量和为实践提供证据基础很有价值。审查常规数据对于审查与社会公正有关的问题,如卫生不平等,可能特别有用。然而,人们对RCD在卫生领域的应用程度以及出版以供更广泛传播的程度知之甚少。目的:本范围审查的目的是记录利用RCD的联合卫生领域同行评议的已发表研究,并评估这些研究在多大程度上解决了与社会公正有关的问题。方法:Westphalm等人提出的Arksey和O'Malley框架的增强版本指导了范围评估。对三个数据库进行了全面的文献检索,确定了1584篇文章。三位研究人员在5%的论文中试用了纳入和排除标准。所有标题和摘要都由2名团队成员独立筛选,全文也是如此。为解决研究问题而开发的数据图表框架由三名研究人员试用,数据提取由首席研究人员完成。论文样本由另一位研究者独立绘制,以进行可靠性检验。结果:共纳入文献190篇。文献中所代表的专业是多种多样的:物理治疗(33.7%)和心理学/精神卫生专业(15.8%)占主导地位。许多研究首先由临床医生(44.2%)撰写,通常由临床-学术团队撰写。有些人(33.25%)直接引用他们的研究来检验研究翻译的实践。很少有研究(14.2%)专门处理与社会正义有关的问题,尽管许多研究收集了可以用于这一目的的变量。结论:将RCD付诸实践的研究可以有效地解决研究与实践之间的差距,并为与社会正义相关的问题提供新的证据。然而,RCD在这些方面没有得到充分利用。鉴于常规收集了大量相关数据,需要做更多的工作来利用它,这将得到对RCD研究价值的更多承认的支持。
{"title":"Operationalising routinely collected patient data in research to further the pursuit of social justice and health equity: a team-based scoping review.","authors":"Katie Chadd, Anna Caute, Anna Pettican, Pam Enderby","doi":"10.1186/s12874-025-02466-9","DOIUrl":"10.1186/s12874-025-02466-9","url":null,"abstract":"<p><strong>Background: </strong>Vast volumes of routinely collected data (RCD) about patients are collated by health professionals. Leveraging this data - a form of real-world data - can be valuable for quality improvement and contributing to the evidence-base to inform practice. Examining routine data may be especially useful for examining issues related to social justice such as health inequities. However, little is known about the extent to which RCD is utilised in health fields and published for wider dissemination.</p><p><strong>Objectives: </strong>The objective of this scoping review is to document the peer-reviewed published research in allied health fields which utilise RCD and evaluate the extent to which these studies have addressed issues pertaining to social justice.</p><p><strong>Methods: </strong>An enhanced version of the Arksey and O'Malley's framework, put forth by Westphalm et al. guided the scoping review. A comprehensive literature search of three databases identified 1584 articles. Application of inclusion and exclusion criteria was piloted on 5% of the papers by three researchers. All titles and abstracts were screened independently by 2 team members, as were full texts. A data charting framework, developed to address the research questions, was piloted by three researchers with data extraction being completed by the lead researcher. A sample of papers were independently charted by a second researcher for reliability checking.</p><p><strong>Results: </strong>One hundred and ninety papers were included in the review. The literature was diverse in terms of the professions that were represented: physiotherapy (33.7%) and psychology/mental health professions (15.8%) predominated. Many studies were first authored by clinicians (44.2%), often with clinical-academic teams. Some (33.25%) directly referenced the use of their studies to examine translation of research to practice. Few studies (14.2%) specifically tackled issues pertaining to social justice, though many collected variables that could have been utilised for this purpose.</p><p><strong>Conclusion: </strong>Studies operationalising RCD can meaningfully address research to practice gaps and provide new evidence about issues related to social justice. However, RCD is underutilised for these purposes. Given that vast volumes of relevant data are routinely collected, more needs to be done to leverage it, which would be supported by greater acknowledgement of the value of RCD studies.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"14"},"PeriodicalIF":3.9,"publicationDate":"2025-01-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11749527/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143000027","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Why and how should we simulate platform trials? Learnings from EU-PEARL. 为什么以及如何模拟平台试验?向EU-PEARL学习。
IF 3.9 3区 医学 Q1 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-01-17 DOI: 10.1186/s12874-024-02453-6
Elias Laurin Meyer, Tobias Mielke, Marta Bofill Roig, Michaela Maria Freitag, Peter Jacko, Pavla Krotka, Peter Mesenbrink, Tom Parke, Sonja Zehetmayer, Dario Zocholl, Franz König

Background: Platform trials are innovative clinical trials governed by a master protocol that allows for the evaluation of multiple investigational treatments that enter and leave the trial over time. Interest in platform trials has been steadily increasing over the last decade. Due to their highly adaptive nature, platform trials provide sufficient flexibility to customize important trial design aspects to the requirements of both the specific disease under investigation and the different stakeholders. The flexibility of platform trials, however, comes with complexities when designing such trials. In the past, we reviewed existing software for simulating clinical trials and found that none of them were suitable for simulating platform trials as they do not accommodate the design features and flexibility inherent to platform trials, such as staggered entry of treatments over time.

Results: We argued that simulation studies are crucial for the design of efficient platform trials. We developed and proposed an iterative, simulation-guided "vanilla and sprinkles" framework, i.e. from a basic to a more complex design, for designing platform trials. We addressed the functionality limitations of existing software as well as the unavailability of the coding therein by developing a suite of open-source software to use in simulating platform trials based on the R programming language. To give some examples, the newly developed software supports simulating staggered entry of treatments throughout the trial, choosing different options for control data sharing, specifying different platform stopping rules and platform-level operating characteristics. The software we developed is available through open-source licensing to enable users to access and modify the code. The separate use of two of these software packages to implement the same platform design by independent teams obtained the same results.

Conclusion: We provide a framework, as well as open-source software for the design and simulation of platform trials. The software tools provide the flexibility necessary to capture the complexity of platform trials.

背景:平台试验是由一个主方案管理的创新临床试验,该方案允许随着时间的推移对进入和退出试验的多种研究性治疗进行评估。在过去的十年里,人们对平台试验的兴趣一直在稳步增长。由于其高度适应性,平台试验提供了足够的灵活性,可以根据所调查的特定疾病和不同利益相关者的要求定制重要的试验设计方面。然而,在设计此类试验时,平台试验的灵活性带来了复杂性。过去,我们回顾了现有的模拟临床试验的软件,发现它们都不适合模拟平台试验,因为它们不适应平台试验固有的设计特征和灵活性,例如随着时间的推移交错进入治疗。结果:我们认为模拟研究对于设计有效的平台试验至关重要。我们开发并提出了一个迭代的、以模拟为指导的“香草和撒糖”框架,即从基本设计到更复杂的设计,用于设计平台试验。我们通过开发一套开源软件来模拟基于R编程语言的平台试验,解决了现有软件的功能限制以及其中编码的不可用性。举例来说,新开发的软件支持在整个试验过程中模拟交错进入治疗方案,选择不同的控制数据共享选项,指定不同的平台停止规则和平台级操作特性。我们开发的软件可以通过开源许可获得,用户可以访问和修改代码。由独立的团队分别使用这两个软件包来实现相同的平台设计,得到了相同的结果。结论:我们为平台试验的设计和仿真提供了一个框架和开源软件。软件工具提供了捕捉平台试验复杂性所需的灵活性。
{"title":"Why and how should we simulate platform trials? Learnings from EU-PEARL.","authors":"Elias Laurin Meyer, Tobias Mielke, Marta Bofill Roig, Michaela Maria Freitag, Peter Jacko, Pavla Krotka, Peter Mesenbrink, Tom Parke, Sonja Zehetmayer, Dario Zocholl, Franz König","doi":"10.1186/s12874-024-02453-6","DOIUrl":"10.1186/s12874-024-02453-6","url":null,"abstract":"<p><strong>Background: </strong>Platform trials are innovative clinical trials governed by a master protocol that allows for the evaluation of multiple investigational treatments that enter and leave the trial over time. Interest in platform trials has been steadily increasing over the last decade. Due to their highly adaptive nature, platform trials provide sufficient flexibility to customize important trial design aspects to the requirements of both the specific disease under investigation and the different stakeholders. The flexibility of platform trials, however, comes with complexities when designing such trials. In the past, we reviewed existing software for simulating clinical trials and found that none of them were suitable for simulating platform trials as they do not accommodate the design features and flexibility inherent to platform trials, such as staggered entry of treatments over time.</p><p><strong>Results: </strong>We argued that simulation studies are crucial for the design of efficient platform trials. We developed and proposed an iterative, simulation-guided \"vanilla and sprinkles\" framework, i.e. from a basic to a more complex design, for designing platform trials. We addressed the functionality limitations of existing software as well as the unavailability of the coding therein by developing a suite of open-source software to use in simulating platform trials based on the R programming language. To give some examples, the newly developed software supports simulating staggered entry of treatments throughout the trial, choosing different options for control data sharing, specifying different platform stopping rules and platform-level operating characteristics. The software we developed is available through open-source licensing to enable users to access and modify the code. The separate use of two of these software packages to implement the same platform design by independent teams obtained the same results.</p><p><strong>Conclusion: </strong>We provide a framework, as well as open-source software for the design and simulation of platform trials. The software tools provide the flexibility necessary to capture the complexity of platform trials.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"12"},"PeriodicalIF":3.9,"publicationDate":"2025-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11740366/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143000042","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A case study in statistical software development for advanced evidence synthesis: the combined value of analysts and research software engineers. 高级证据合成的统计软件开发案例研究:分析人员和研究软件工程师的综合价值。
IF 3.9 3区 医学 Q1 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-01-17 DOI: 10.1186/s12874-024-02450-9
Naomi Bradbury, Tom Morris, Clareece Nevill, Janion Nevill, Ryan Field, Suzanne Freeman, Nicola Cooper, Alex Sutton

Background: Since 2015, the Complex Reviews Synthesis Unit (CRSU) has developed a suite of web-based applications (apps) that conduct complex evidence synthesis meta-analyses through point-and-click interfaces. This has been achieved in the R programming language by combining existing R packages that conduct meta-analysis with the shiny web-application package. The CRSU apps have evolved from two short-term student projects into a suite of eight apps that are used for more than 3,000 h per month.

Aim: Here, we present our experience of developing production grade web-apps from the point-of-view of individuals trained primarily as statisticians rather than software developers in the hopes of encouraging and inspiring other groups to develop valuable open-source statistical software whilst also learning from our experiences.

Key challenges: We discuss how we have addressed challenges to research software development such as responding to feedback from our real-world users to improve the CRSU apps, the implementation of software engineering principles into our app development process and gaining recognition for non-traditional research work within the academic environment.

Future developments: The CRSU continues to seek funding opportunities both to maintain and further develop our shiny apps. We aim to increase our user base by implementing new features within the apps and building links with other groups developing complementary evidence synthesis tools.

背景:自2015年以来,复杂评论综合单元(CRSU)开发了一套基于网络的应用程序(app),通过点击界面进行复杂证据综合元分析。这是在R编程语言中实现的,通过将现有的R包与闪亮的web应用程序包结合起来进行元分析。CRSU的应用程序已经从两个短期的学生项目发展成为一个包含八个应用程序的套件,每个月的使用时间超过3000小时。目的:在这里,我们从主要作为统计学家而不是软件开发人员的个人角度来介绍我们开发生产级web应用程序的经验,希望鼓励和激励其他团体开发有价值的开源统计软件,同时也从我们的经验中学习。主要挑战:我们讨论了我们如何应对研究软件开发的挑战,例如响应现实世界用户的反馈以改进CRSU应用程序,将软件工程原则实施到我们的应用程序开发过程中,以及在学术环境中获得非传统研究工作的认可。未来发展:CRSU将继续寻求融资机会,以维护和进一步开发我们闪亮的应用程序。我们的目标是通过在应用程序中实现新功能并与其他开发补充证据合成工具的小组建立联系来增加我们的用户基础。
{"title":"A case study in statistical software development for advanced evidence synthesis: the combined value of analysts and research software engineers.","authors":"Naomi Bradbury, Tom Morris, Clareece Nevill, Janion Nevill, Ryan Field, Suzanne Freeman, Nicola Cooper, Alex Sutton","doi":"10.1186/s12874-024-02450-9","DOIUrl":"10.1186/s12874-024-02450-9","url":null,"abstract":"<p><strong>Background: </strong>Since 2015, the Complex Reviews Synthesis Unit (CRSU) has developed a suite of web-based applications (apps) that conduct complex evidence synthesis meta-analyses through point-and-click interfaces. This has been achieved in the R programming language by combining existing R packages that conduct meta-analysis with the shiny web-application package. The CRSU apps have evolved from two short-term student projects into a suite of eight apps that are used for more than 3,000 h per month.</p><p><strong>Aim: </strong>Here, we present our experience of developing production grade web-apps from the point-of-view of individuals trained primarily as statisticians rather than software developers in the hopes of encouraging and inspiring other groups to develop valuable open-source statistical software whilst also learning from our experiences.</p><p><strong>Key challenges: </strong>We discuss how we have addressed challenges to research software development such as responding to feedback from our real-world users to improve the CRSU apps, the implementation of software engineering principles into our app development process and gaining recognition for non-traditional research work within the academic environment.</p><p><strong>Future developments: </strong>The CRSU continues to seek funding opportunities both to maintain and further develop our shiny apps. We aim to increase our user base by implementing new features within the apps and building links with other groups developing complementary evidence synthesis tools.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"13"},"PeriodicalIF":3.9,"publicationDate":"2025-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11740572/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142999799","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Development, validation, and usage of metrics to evaluate the quality of clinical research hypotheses. 开发、验证和使用指标来评估临床研究假设的质量。
IF 3.9 3区 医学 Q1 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-01-16 DOI: 10.1186/s12874-025-02460-1
Xia Jing, Yuchun Zhou, James J Cimino, Jay H Shubrook, Vimla L Patel, Sonsoles De Lacalle, Aneesa Weaver, Chang Liu

Objectives: Metrics and instruments can provide guidance for clinical researchers to assess their potential research projects at an early stage before significant investment. Furthermore, metrics can also provide structured criteria for peer reviewers to assess others' clinical research manuscripts or grant proposals. This study aimed to develop, test, validate, and use evaluation metrics and instruments to accurately, consistently, systematically, and conveniently assess the quality of scientific hypotheses for clinical research projects.

Materials and methods: Metrics development went through iterative stages, including literature review, metrics and instrument development, internal and external testing and validation, and continuous revisions in each stage based on feedback. Furthermore, two experiments were conducted to determine brief and comprehensive versions of the instrument.

Results: The brief version of the instrument contained three dimensions: validity, significance, and feasibility. The comprehensive version of metrics included novelty, clinical relevance, potential benefits and risks, ethicality, testability, clarity, interestingness, and the three dimensions of the brief version. Each evaluation dimension included 2 to 5 subitems to evaluate the specific aspects of each dimension. For example, validity included clinical validity and scientific validity. The brief and comprehensive versions of the instruments included 12 and 39 subitems, respectively. Each subitem used a 5-point Likert scale.

Conclusion: The validated brief and comprehensive versions of metrics can provide standardized, consistent, systematic, and generic measurements for clinical research hypotheses, allow clinical researchers to prioritize their research ideas systematically, objectively, and consistently, and can be used as a tool for quality assessment during the peer review process.

目的:指标和工具可为临床研究人员在重大投资前早期评估其潜在研究项目提供指导。此外,度量标准还可以为同行审稿人提供结构化的标准,以评估他人的临床研究手稿或资助提案。本研究旨在开发、测试、验证和使用评估指标和工具,以准确、一致、系统和方便地评估临床研究项目的科学假设的质量。材料和方法:度量开发经历了迭代阶段,包括文献回顾、度量和仪器开发、内部和外部测试和验证,以及基于反馈的每个阶段的持续修订。此外,还进行了两次实验,以确定该仪器的简要和全面版本。结果:简易版量表包含三个维度:效度、意义和可行性。综合版本的指标包括新颖性、临床相关性、潜在益处和风险、伦理性、可测试性、清晰度、趣味性和简要版本的三个维度。每个评价维度包括2 - 5个子项,以评价每个维度的具体方面。例如,效度包括临床效度和科学效度。文书的简要本和综合本分别包括12个和39个分项。每个子项采用5分李克特量表。结论:经过验证的简短、全面的指标版本可以为临床研究假设提供标准化、一致、系统和通用的测量,使临床研究人员能够系统、客观、一致地优先考虑他们的研究想法,并可作为同行评议过程中质量评估的工具。
{"title":"Development, validation, and usage of metrics to evaluate the quality of clinical research hypotheses.","authors":"Xia Jing, Yuchun Zhou, James J Cimino, Jay H Shubrook, Vimla L Patel, Sonsoles De Lacalle, Aneesa Weaver, Chang Liu","doi":"10.1186/s12874-025-02460-1","DOIUrl":"10.1186/s12874-025-02460-1","url":null,"abstract":"<p><strong>Objectives: </strong>Metrics and instruments can provide guidance for clinical researchers to assess their potential research projects at an early stage before significant investment. Furthermore, metrics can also provide structured criteria for peer reviewers to assess others' clinical research manuscripts or grant proposals. This study aimed to develop, test, validate, and use evaluation metrics and instruments to accurately, consistently, systematically, and conveniently assess the quality of scientific hypotheses for clinical research projects.</p><p><strong>Materials and methods: </strong>Metrics development went through iterative stages, including literature review, metrics and instrument development, internal and external testing and validation, and continuous revisions in each stage based on feedback. Furthermore, two experiments were conducted to determine brief and comprehensive versions of the instrument.</p><p><strong>Results: </strong>The brief version of the instrument contained three dimensions: validity, significance, and feasibility. The comprehensive version of metrics included novelty, clinical relevance, potential benefits and risks, ethicality, testability, clarity, interestingness, and the three dimensions of the brief version. Each evaluation dimension included 2 to 5 subitems to evaluate the specific aspects of each dimension. For example, validity included clinical validity and scientific validity. The brief and comprehensive versions of the instruments included 12 and 39 subitems, respectively. Each subitem used a 5-point Likert scale.</p><p><strong>Conclusion: </strong>The validated brief and comprehensive versions of metrics can provide standardized, consistent, systematic, and generic measurements for clinical research hypotheses, allow clinical researchers to prioritize their research ideas systematically, objectively, and consistently, and can be used as a tool for quality assessment during the peer review process.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"11"},"PeriodicalIF":3.9,"publicationDate":"2025-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11737058/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143000002","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Efficient evidence selection for systematic reviews in traditional Chinese medicine. 中医系统评价的有效证据选择。
IF 3.9 3区 医学 Q1 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-01-15 DOI: 10.1186/s12874-024-02430-z
Yizhen Li, Zhe Huang, Zhongzhi Luan, Shujing Xu, Yunan Zhang, Lin Wu, Darong Wu, Dongran Han, Yixing Liu

Purpose: The process of searching for and selecting clinical evidence for systematic reviews (SRs) or clinical guidelines is essential for researchers in Traditional Chinese medicine (TCM). However, this process is often time-consuming and resource-intensive. In this study, we introduce a novel precision-preferred comprehensive information extraction and selection procedure to enhance both the efficiency and accuracy of evidence selection for TCM practitioners.

Methods: We integrated an established deep learning model (Evi-BERT combined rule-based method) with Boolean logic algorithms and an expanded retrieval strategy to automatically and accurately select potential evidence with minimal human intervention. The selection process is recorded in real-time, allowing researchers to backtrack and verify its accuracy. This innovative approach was tested on ten high-quality, randomly selected systematic reviews of TCM-related topics written in Chinese. To evaluate its effectiveness, we compared the screening time and accuracy of this approach with traditional evidence selection methods.

Results: Our finding demonstrated that the new method accurately selected potential literature based on consistent criteria while significantly reducing the time required for the process. Additionally, in some cases, this approach identified a broader range of relevant evidence and enabled the tracking of selection progress for future reference. The study also revealed that traditional screening methods are often subjective and prone to errors, frequently resulting in the inclusion of literature that does not meet established standards. In contrast, our method offers a more accurate and efficient way to select clinical evidence for TCM practitioners, outperforming traditional manual approaches.

Conclusion: We proposed an innovative approach for selecting clinical evidence for TCM reviews and guidelines, aiming to reduce the workload for researchers. While this method showed promise in improving the efficiency and accuracy of evidence-based selection, its full potential required further validation. Additionally, it may serve as a useful tool for editors to assess manuscript quality in the future.

目的:系统评价(SRs)或临床指南的临床证据的检索和选择是中医研究人员必不可少的过程。然而,这个过程通常是耗时和资源密集的。在本研究中,我们引入了一种新的精确优先的综合信息提取和选择程序,以提高中医医生证据选择的效率和准确性。方法:我们将建立的深度学习模型(Evi-BERT结合基于规则的方法)与布尔逻辑算法和扩展的检索策略相结合,在最小的人为干预下自动准确地选择潜在证据。选择过程是实时记录的,允许研究人员回溯并验证其准确性。这个创新的方法在十篇用中文写的高质量的、随机选择的中医相关主题的系统综述中进行了测试。为了评估其有效性,我们比较了该方法与传统证据选择方法的筛选时间和准确性。结果:新方法能够根据一致的标准准确地选择潜在文献,同时显著缩短了过程所需的时间。此外,在某些情况下,这种方法确定了更广泛的相关证据,并能够跟踪选择过程,以供将来参考。该研究还表明,传统的筛选方法往往是主观的,容易出错,经常导致纳入不符合既定标准的文献。相比之下,我们的方法为中医医生提供了一种更准确、更有效的临床证据选择方法,优于传统的手工方法。结论:我们提出了一种创新的方法来选择中医评论和指南的临床证据,旨在减少研究人员的工作量。虽然该方法有望提高循证选择的效率和准确性,但其全部潜力需要进一步验证。此外,它可以作为一个有用的工具,为编辑评估手稿的质量在未来。
{"title":"Efficient evidence selection for systematic reviews in traditional Chinese medicine.","authors":"Yizhen Li, Zhe Huang, Zhongzhi Luan, Shujing Xu, Yunan Zhang, Lin Wu, Darong Wu, Dongran Han, Yixing Liu","doi":"10.1186/s12874-024-02430-z","DOIUrl":"10.1186/s12874-024-02430-z","url":null,"abstract":"<p><strong>Purpose: </strong>The process of searching for and selecting clinical evidence for systematic reviews (SRs) or clinical guidelines is essential for researchers in Traditional Chinese medicine (TCM). However, this process is often time-consuming and resource-intensive. In this study, we introduce a novel precision-preferred comprehensive information extraction and selection procedure to enhance both the efficiency and accuracy of evidence selection for TCM practitioners.</p><p><strong>Methods: </strong>We integrated an established deep learning model (Evi-BERT combined rule-based method) with Boolean logic algorithms and an expanded retrieval strategy to automatically and accurately select potential evidence with minimal human intervention. The selection process is recorded in real-time, allowing researchers to backtrack and verify its accuracy. This innovative approach was tested on ten high-quality, randomly selected systematic reviews of TCM-related topics written in Chinese. To evaluate its effectiveness, we compared the screening time and accuracy of this approach with traditional evidence selection methods.</p><p><strong>Results: </strong>Our finding demonstrated that the new method accurately selected potential literature based on consistent criteria while significantly reducing the time required for the process. Additionally, in some cases, this approach identified a broader range of relevant evidence and enabled the tracking of selection progress for future reference. The study also revealed that traditional screening methods are often subjective and prone to errors, frequently resulting in the inclusion of literature that does not meet established standards. In contrast, our method offers a more accurate and efficient way to select clinical evidence for TCM practitioners, outperforming traditional manual approaches.</p><p><strong>Conclusion: </strong>We proposed an innovative approach for selecting clinical evidence for TCM reviews and guidelines, aiming to reduce the workload for researchers. While this method showed promise in improving the efficiency and accuracy of evidence-based selection, its full potential required further validation. Additionally, it may serve as a useful tool for editors to assess manuscript quality in the future.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"10"},"PeriodicalIF":3.9,"publicationDate":"2025-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11734327/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143000018","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Methodological challenges using routine clinical care data for real-world evidence: a rapid review utilizing a systematic literature search and focus group discussion. 使用常规临床护理数据获取真实世界证据的方法学挑战:利用系统文献检索和焦点小组讨论的快速回顾。
IF 3.9 3区 医学 Q1 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-01-14 DOI: 10.1186/s12874-024-02440-x
Michelle Pfaffenlehner, Max Behrens, Daniela Zöller, Kathrin Ungethüm, Kai Günther, Viktoria Rücker, Jens-Peter Reese, Peter Heuschmann, Miriam Kesselmeier, Flavia Remo, André Scherag, Harald Binder, Nadine Binder

Background: The integration of real-world evidence (RWE) from real-world data (RWD) in clinical research is crucial for bridging the gap between clinical trial results and real-world outcomes. Analyzing routinely collected data to generate clinical evidence faces methodological concerns like confounding and bias, similar to prospectively documented observational studies. This study focuses on additional limitations frequently reported in the literature, providing an overview of the challenges and biases inherent to analyzing routine clinical care data, including health claims data (hereafter: routine data).

Methods: We conducted a literature search on routine data studies in four high-impact journals based on the Journal Citation Reports (JCR) category "Medicine, General & Internal" as of 2022 and three oncology journals, covering articles published from January 2018 to October 2023. Articles were screened and categorized into three scenarios based on their potential to provide meaningful RWE: (1) Burden of Disease, (2) Safety and Risk Group Analysis, and (3) Treatment Comparison. Limitations of this type of data cited in the discussion sections were extracted and classified according to different bias types: main bias categories in non-randomized studies (information bias, reporting bias, selection bias, confounding) and additional routine data-specific challenges (i.e., operationalization, coding, follow-up, missing data, validation, and data quality). These classifications were then ranked by relevance in a focus group meeting of methodological experts. The search was pre-specified and registered in PROSPERO (CRD42023477616).

Results: In October 2023, 227 articles were identified, 69 were assessed for eligibility, and 39 were included in the review: 11 on the burden of disease, 17 on safety and risk group analysis, and 11 on treatment comparison. Besides typical biases in observational studies, we identified additional challenges specific to RWE frequently mentioned in the discussion sections. The focus group had varied opinions on the limitations of Safety and Risk Group Analysis and Treatment Comparison but agreed on the essential limitations for the Burden of Disease category.

Conclusion: This review provides a comprehensive overview of potential limitations and biases in analyzing routine data reported in recent high-impact journals. We highlighted key challenges that have high potential to impact analysis results, emphasizing the need for thorough consideration and discussion for meaningful inferences.

背景:临床研究中真实世界证据(RWE)与真实世界数据(RWD)的整合对于弥合临床试验结果与真实世界结果之间的差距至关重要。分析常规收集的数据以产生临床证据面临混淆和偏倚等方法学问题,类似于前瞻性记录的观察性研究。本研究着重于文献中经常报道的其他限制,概述了分析常规临床护理数据(包括健康声明数据)所固有的挑战和偏见。方法:检索截至2022年JCR期刊引文报告(Journal Citation Reports)“Medicine, General & Internal”类别的4种高影响力期刊和3种肿瘤学期刊的常规数据研究,涵盖2018年1月至2023年10月发表的文章。文章根据其提供有意义RWE的潜力被筛选并分为三种情景:(1)疾病负担,(2)安全性和风险组分析,以及(3)治疗比较。根据不同的偏倚类型提取和分类讨论部分中引用的这类数据的局限性:非随机研究中的主要偏倚类别(信息偏倚、报告偏倚、选择偏倚、混淆)和额外的常规数据特定挑战(即操作化、编码、随访、缺失数据、验证和数据质量)。然后在方法学专家焦点小组会议上按相关性对这些分类进行排序。该搜索在PROSPERO (CRD42023477616)中预先指定并注册。结果:在2023年10月,227篇文章被识别,69篇被评估为合格,39篇被纳入审查:11篇关于疾病负担,17篇关于安全性和风险组分析,11篇关于治疗比较。除了观察性研究中的典型偏差外,我们还确定了讨论部分经常提到的RWE特有的其他挑战。焦点小组对安全性和风险组分析和治疗比较的局限性有不同的意见,但对疾病负担类别的基本局限性达成一致。结论:本综述全面概述了在分析近期高影响力期刊报道的常规数据时可能存在的局限性和偏倚。我们强调了对分析结果有很大影响的关键挑战,强调需要对有意义的推论进行彻底的考虑和讨论。
{"title":"Methodological challenges using routine clinical care data for real-world evidence: a rapid review utilizing a systematic literature search and focus group discussion.","authors":"Michelle Pfaffenlehner, Max Behrens, Daniela Zöller, Kathrin Ungethüm, Kai Günther, Viktoria Rücker, Jens-Peter Reese, Peter Heuschmann, Miriam Kesselmeier, Flavia Remo, André Scherag, Harald Binder, Nadine Binder","doi":"10.1186/s12874-024-02440-x","DOIUrl":"10.1186/s12874-024-02440-x","url":null,"abstract":"<p><strong>Background: </strong>The integration of real-world evidence (RWE) from real-world data (RWD) in clinical research is crucial for bridging the gap between clinical trial results and real-world outcomes. Analyzing routinely collected data to generate clinical evidence faces methodological concerns like confounding and bias, similar to prospectively documented observational studies. This study focuses on additional limitations frequently reported in the literature, providing an overview of the challenges and biases inherent to analyzing routine clinical care data, including health claims data (hereafter: routine data).</p><p><strong>Methods: </strong>We conducted a literature search on routine data studies in four high-impact journals based on the Journal Citation Reports (JCR) category \"Medicine, General & Internal\" as of 2022 and three oncology journals, covering articles published from January 2018 to October 2023. Articles were screened and categorized into three scenarios based on their potential to provide meaningful RWE: (1) Burden of Disease, (2) Safety and Risk Group Analysis, and (3) Treatment Comparison. Limitations of this type of data cited in the discussion sections were extracted and classified according to different bias types: main bias categories in non-randomized studies (information bias, reporting bias, selection bias, confounding) and additional routine data-specific challenges (i.e., operationalization, coding, follow-up, missing data, validation, and data quality). These classifications were then ranked by relevance in a focus group meeting of methodological experts. The search was pre-specified and registered in PROSPERO (CRD42023477616).</p><p><strong>Results: </strong>In October 2023, 227 articles were identified, 69 were assessed for eligibility, and 39 were included in the review: 11 on the burden of disease, 17 on safety and risk group analysis, and 11 on treatment comparison. Besides typical biases in observational studies, we identified additional challenges specific to RWE frequently mentioned in the discussion sections. The focus group had varied opinions on the limitations of Safety and Risk Group Analysis and Treatment Comparison but agreed on the essential limitations for the Burden of Disease category.</p><p><strong>Conclusion: </strong>This review provides a comprehensive overview of potential limitations and biases in analyzing routine data reported in recent high-impact journals. We highlighted key challenges that have high potential to impact analysis results, emphasizing the need for thorough consideration and discussion for meaningful inferences.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"8"},"PeriodicalIF":3.9,"publicationDate":"2025-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11731536/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142982630","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Recruiting participants for focus groups in health research: a meta-research study. 招募健康研究焦点小组的参与者:一项元研究研究。
IF 3.9 3区 医学 Q1 HEALTH CARE SCIENCES & SERVICES Pub Date : 2025-01-14 DOI: 10.1186/s12874-025-02464-x
Jonas Lander, Simon Wallraf, Dawid Pieper, Ronny Klawunn, Hala Altawil, Marie-Luise Dierks, Cosima John

Background: Focus groups (FGs) are an established method in health research to capture a full range of different perspectives on a particular research question. The extent to which they are effective depends, not least, on the composition of the participants. This study aimed to investigate how published FG studies plan and conduct the recruitment of study participants. We looked at what kind of information is reported about recruitment practices and what this reveals about the comprehensiveness of the actual recruitment plans and practices.

Methods: We conducted a systematic search of FG studies in PubMed and Web of Science published between 2018 and 2024, and included n = 80 eligible publications in the analysis. We used a text extraction sheet to collect all relevant recruitment information from each study. We then coded the extracted text passages and summarised the findings descriptively.

Results: Nearly half (n = 38/80) of the studies were from the USA and Canada, many addressing issues related to diabetes, cancer, mental health and chronic diseases. For recruitment planning, 20% reported a specific sampling target, while 6% used existing studies or literature for organisational and content planning. A further 10% reported previous recruitment experience of the researchers. The studies varied in terms of number of participants (range = 7-202) and group size (range = 7-20). Recruitment occurred often in healthcare settings, rarely through digital channels and everyday places. FG participants were most commonly recruited by the research team (21%) or by health professionals (16%), with less collaboration with public organisations (10%) and little indication of the number of people involved (13%). A financial incentive for participants was used in 43% of cases, and 19% reported participatory approaches to plan and carry out recruitment. 65 studies (81%) reported a total of 58 limitations related to recruitment.

Conclusions: The reporting of recruitment often seems to be incomplete, and its performance lacking. Hence, guidelines and recruitment recommendations designed to assist researchers are not yet adequately serving their purpose. Researchers may benefit from more practical support, such as early training on key principles and options for effective recruitment strategies provided by institutions in their immediate professional environment, e.g. universities, faculties or scientific associations.

背景:焦点小组(FGs)是卫生研究中的一种既定方法,用于捕获对特定研究问题的全方位不同观点。它们的有效程度不仅取决于参与者的构成。本研究旨在探讨已发表的FG研究如何计划和招募研究参与者。我们研究了关于招聘实践的哪些信息被报道,以及这些信息揭示了实际招聘计划和实践的全面性。方法:系统检索2018年至2024年间发表在PubMed和Web of Science上的FG研究,纳入n = 80篇符合条件的论文。我们使用文本提取表收集每个研究的所有相关招募信息。然后,我们对提取的文本段落进行编码,并对结果进行描述性总结。结果:近一半(n = 38/80)的研究来自美国和加拿大,许多研究涉及与糖尿病、癌症、心理健康和慢性病相关的问题。对于招聘计划,20%的人报告了一个特定的抽样目标,而6%的人使用现有的研究或文献进行组织和内容规划。另有10%的人报告了之前招募研究人员的经历。这些研究在参与者人数(范围= 7-202)和小组规模(范围= 7-20)方面有所不同。招聘通常发生在医疗机构,很少通过数字渠道和日常场所。FG参与者通常是由研究小组(21%)或卫生专业人员(16%)招募的,与公共组织的合作较少(10%),很少表明参与人数(13%)。43%的案例采用了对参与者的经济激励,19%的案例采用了参与式方法来计划和实施招聘。65项研究(81%)报告了与招募相关的58项限制。结论:招聘的报道往往显得不完整,缺乏实效性。因此,旨在帮助研究人员的指导方针和招聘建议尚未充分服务于其目的。研究人员可以从更实际的支持中受益,例如在其直接的专业环境中,如大学、学院或科学协会,机构提供的关于关键原则和有效招聘策略选择的早期培训。
{"title":"Recruiting participants for focus groups in health research: a meta-research study.","authors":"Jonas Lander, Simon Wallraf, Dawid Pieper, Ronny Klawunn, Hala Altawil, Marie-Luise Dierks, Cosima John","doi":"10.1186/s12874-025-02464-x","DOIUrl":"10.1186/s12874-025-02464-x","url":null,"abstract":"<p><strong>Background: </strong>Focus groups (FGs) are an established method in health research to capture a full range of different perspectives on a particular research question. The extent to which they are effective depends, not least, on the composition of the participants. This study aimed to investigate how published FG studies plan and conduct the recruitment of study participants. We looked at what kind of information is reported about recruitment practices and what this reveals about the comprehensiveness of the actual recruitment plans and practices.</p><p><strong>Methods: </strong>We conducted a systematic search of FG studies in PubMed and Web of Science published between 2018 and 2024, and included n = 80 eligible publications in the analysis. We used a text extraction sheet to collect all relevant recruitment information from each study. We then coded the extracted text passages and summarised the findings descriptively.</p><p><strong>Results: </strong>Nearly half (n = 38/80) of the studies were from the USA and Canada, many addressing issues related to diabetes, cancer, mental health and chronic diseases. For recruitment planning, 20% reported a specific sampling target, while 6% used existing studies or literature for organisational and content planning. A further 10% reported previous recruitment experience of the researchers. The studies varied in terms of number of participants (range = 7-202) and group size (range = 7-20). Recruitment occurred often in healthcare settings, rarely through digital channels and everyday places. FG participants were most commonly recruited by the research team (21%) or by health professionals (16%), with less collaboration with public organisations (10%) and little indication of the number of people involved (13%). A financial incentive for participants was used in 43% of cases, and 19% reported participatory approaches to plan and carry out recruitment. 65 studies (81%) reported a total of 58 limitations related to recruitment.</p><p><strong>Conclusions: </strong>The reporting of recruitment often seems to be incomplete, and its performance lacking. Hence, guidelines and recruitment recommendations designed to assist researchers are not yet adequately serving their purpose. Researchers may benefit from more practical support, such as early training on key principles and options for effective recruitment strategies provided by institutions in their immediate professional environment, e.g. universities, faculties or scientific associations.</p>","PeriodicalId":9114,"journal":{"name":"BMC Medical Research Methodology","volume":"25 1","pages":"9"},"PeriodicalIF":3.9,"publicationDate":"2025-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11730470/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142982650","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
BMC Medical Research Methodology
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1