Journal of Clinical Epidemiology最新文献

{"title":"The interplay between PROM score distributions and treatment effect detection likelihood in randomized controlled trials–a metaepidemiologic study","authors":"Valtteri Panula ,&nbsp;Antti Saarinen ,&nbsp;Matias Vaajala ,&nbsp;Rasmus Liukkonen ,&nbsp;Oskari Pakarinen ,&nbsp;Juho Laaksonen ,&nbsp;Ville Ponkilainen ,&nbsp;Ilari Kuitunen ,&nbsp;Mikko Uimonen","doi":"10.1016/j.jclinepi.2025.112114","DOIUrl":"10.1016/j.jclinepi.2025.112114","url":null,"abstract":"<div><h3>Objectives</h3><div>We hypothesized that, in musculoskeletal randomized controlled trials (RCTs) using patient-reported outcome measures (PROMs), higher baseline scores and the clustering of follow-up scores near the upper bound (ie, ceiling effect) compress variability and attenuate measurable between-group differences, thereby lowering the likelihood of observing a statistically significant effect. We therefore examined how score distributions at pretreatment and follow-up influence the likelihood of detecting between-group differences.</div></div><div><h3>Study Design and Setting</h3><div>We conducted a metaepidemiologic study of RCTs, published between 2015 and 2024, that compared treatment effects on musculoskeletal disorders between two study groups using PROMs. The observed distributions of the PROM scores at baseline and follow-up were collected from the included studies. All PROM scores were rescaled to 0-100 with higher scores indicating better health. The likelihood of observing a statistically significant difference in PROM scores between the study groups was examined by calculating the score difference required to achieve a <em>P</em> value &lt;.05.</div></div><div><h3>Results</h3><div>A total of 255 RCTs were included. PROM scores improved from baseline to follow-up in most studies (98%), with a mean change of +28 points. The correlation coefficient between the mean baseline score and mean score change was −0.66 (95% CI -0.72 to −0.59) indicating that higher baseline scores were associated with lower score change. In addition, there was a moderate correlation between the mean and SD of PROM scores at follow-up (−0.39; 95% CI -0.48 to −0.28). The mean likelihood of detecting a difference was 65% (SD 11%) at baseline and 65% (SD 11%) at follow-up. The likelihood reached the 80% benchmark in only 8.5% and 8.1% of the studies at baseline and follow-up, respectively.</div></div><div><h3>Conclusion</h3><div>The concentration of PROM score distributions toward the high end of the scale, especially when higher baseline scores are present, diminishes the likelihood of detecting significant differences between study groups, particularly at follow-up assessments in studies analyzing musculoskeletal complaints. This underscores the importance of critically evaluating the conclusions drawn from these studies.</div></div>","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"191 ","pages":"Article 112114"},"PeriodicalIF":5.2,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145806149","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The use of guidelines in multimorbidity related practice: An exploratory questionnaire survey.","authors":"Zijun Wang, Hongfeng He, Sergey K Zyryanov, Liliya E Ziganshina, Akihiko Ozaki, Natalia Dorofeeva, Myeong Soo Lee, Ivan D Florez, Etienne Ngeh, Abhilasha Sharma, Ekaterina V Yudina, Barbara C van Munster, Jako S Burgers, Opeyemi O Babatunde, Yaolong Chen, Janne Estill","doi":"10.1016/j.jclinepi.2025.112115","DOIUrl":"https://doi.org/10.1016/j.jclinepi.2025.112115","url":null,"abstract":"<p><strong>Introduction: </strong>The use of guidelines in multimorbidity-related practice has not yet been extensively investigated. We aimed to explore how healthcare professionals use guidelines when managing individuals with multimorbidity.</p><p><strong>Method: </strong>We conducted an exploratory survey among a convenience sample of medical professionals with clinical experience. The questionnaire addressed whether and how different types of guidelines are used in multimorbidity-related practice, the reasons for not using specific types of guidelines, and other approaches to inform multimorbidity practice. It was distributed through the investigators' contact networks. The results were presented descriptively.</p><p><strong>Result: </strong>We received 311 valid responses: 136 from the WHO European Region, 137 from the Western Pacific Region, and 38 from other regions. Most participants were familiar with the concept of multimorbidity (n=245, 79%). Among the 269 respondents who reported using guidelines in multimorbidity practice, 124 (46%) used guidelines specifically focusing on combinations of diseases, and 148 (55%) multiple single-disease guidelines together. Lack of availability was the main reason for not using guidelines that address multimorbidity itself; and the high number of guidelines (n=76, 40%) and possible interactions between conditions or treatments (n=62, 38%) for not using single-disease guidelines. Respondents frequently consult experts or refer to systematic reviews and primary studies when existing guidelines do not meet their needs. The development of a tool or method to guide the use of multiple guidelines ranked highest among possible actions to improve multimorbidity practice.</p><p><strong>Conclusion: </strong>Although the medical professionals in our sample were generally familiar with use of guidelines, there are many unmet needs and tool gaps related to guideline-informed multimorbidity-related practice.</p>","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":" ","pages":"112115"},"PeriodicalIF":5.2,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145806152","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Sequential sample size calculations and learning curves safeguard the robust development of a clinical prediction model for individuals","authors":"Amardeep Legha ,&nbsp;Joie Ensor ,&nbsp;Rebecca Whittle ,&nbsp;Lucinda Archer ,&nbsp;Ben Van Calster ,&nbsp;Evangelia Christodoulou ,&nbsp;Kym I.E. Snell ,&nbsp;Mohsen Sadatsafavi ,&nbsp;Gary S. Collins ,&nbsp;Richard D. Riley","doi":"10.1016/j.jclinepi.2025.112117","DOIUrl":"10.1016/j.jclinepi.2025.112117","url":null,"abstract":"<div><h3>Background and Objectives</h3><div>When recruiting participants to a new study developing a clinical prediction model (CPM), sample size calculations are typically conducted before data collection based on sensible assumptions. This leads to a fixed sample size, but if the assumptions are inaccurate, the actual sample size required to develop a reliable model may be higher or even lower. To safeguard against this, adaptive sample size approaches have been proposed, based on sequential evaluation of (changes in) a model's predictive performance. The objective of the study was to illustrate and extend sequential sample size calculations for CPM development by (i) proposing stopping rules for prospective data collection based on minimizing uncertainty (instability) and misclassification of individual-level predictions and (ii) showcasing how it safeguards against inaccurate fixed sample size calculations.</div></div><div><h3>Methods</h3><div>Using the sequential approach repeats the predefined model development strategy every time a chosen number (eg, 100) of participants are recruited and adequately followed up. At each stage, CPM performance is evaluated using bootstrapping, leading to prediction and classification stability statistics and plots, alongside optimism-adjusted measures of calibration and discrimination. Learning curves display the trend of results against sample size and recruitment is stopped when a chosen stopping rule is met.</div></div><div><h3>Results</h3><div>Our approach is illustrated for model development of acute kidney injury using (penalized) logistic regression CPMs. Before recruitment based on perceived sensible assumptions, the fixed sample size calculation suggests recruiting 342 patients to minimize overfitting; however, during data collection, the sequential approach reveals that a much larger sample size of 1100 is required to minimize overfitting (targeting a bootstrap-corrected calibration slope ≥0.9). If the stopping rule criteria also target small uncertainty and misclassification probability of individual predictions, the sequential approach suggests an even larger sample size of about 1800.</div></div><div><h3>Conclusion</h3><div>For CPM development studies involving prospective data collection, a sequential sample size approach allows users to dynamically monitor individual-level prediction and classification instability. This helps determine when enough participants have been recruited and safeguards against using inaccurate assumptions in a sample size calculation before data collection. Engagement with patients and other stakeholders is crucial to identify sensible context-specific stopping rules for robust individual predictions.</div></div>","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"191 ","pages":"Article 112117"},"PeriodicalIF":5.2,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145806008","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Text message incentives increased patient-reported outcomes survey response in emergency care: SWAT findings","authors":"Gemma Altinger ,&nbsp;Sweekriti Sharma ,&nbsp;Qiang Li ,&nbsp;Anthony Devaux ,&nbsp;Samantha Darby ,&nbsp;Aidan van Wyk ,&nbsp;Caitlin M.P. Jones ,&nbsp;Chris G. Maher ,&nbsp;Adrian C. Traeger","doi":"10.1016/j.jclinepi.2025.112116","DOIUrl":"10.1016/j.jclinepi.2025.112116","url":null,"abstract":"&lt;div&gt;&lt;h3&gt;Objectives&lt;/h3&gt;&lt;div&gt;To determine if text message–based behavioral interventions could increase response rates to a patient-reported outcomes survey in the emergency department (ED).&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Study Design and Setting&lt;/h3&gt;&lt;div&gt;We conducted a study within a trial (SWAT), within the NUDG-ED trial. The NUDG-ED trial aimed to reduce low-value care for patients with back pain presenting to eight EDs in Sydney, Australia. This SWAT was a 3-arm randomized controlled trial (RCT) nested within the NUDG-ED trial. After discharge from the ED, patients were randomized to receive one of the three text message invitations to complete a follow-up patient-reported outcome survey: a standard control message, or one of two behaviorally informed messages including either a prize draw incentive or prosocial framing. Our primary outcome measure was the response rate in each study group. We performed a linear mixed-effects model controlling for hospital heterogeneity and patient characteristics to estimate the mean difference (MD) in proportions with 95% CI, to determine the effectiveness of the behavioral interventions.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Results&lt;/h3&gt;&lt;div&gt;A total of 1494 patients were randomized between May 15, 2024 and January 29, 2025. Of these, 52% were women, the median age was 46 years (IQR 35, 62), 43% were from disadvantaged areas and 51% were triaged with a clinically urgent condition. Baseline characteristics were balanced across all groups. Our primary analysis found that compared to the control, the prize draw incentive increased response rates (&lt;em&gt;n&lt;/em&gt; = 997 patients, MD = 6.9%, 95% CI: 1.8% to 11.9%, &lt;em&gt;P&lt;/em&gt; = .007). Our adjusted mixed-effects model also found a significant increase in response rates (&lt;em&gt;n&lt;/em&gt; = 979 patients, MD = 6.4%, 95% CI 1.3% to 11.4%, &lt;em&gt;P&lt;/em&gt; = .013). Compared to the control, the prosocial framing message may have slightly increased response rates, but the results were not statistically significant (&lt;em&gt;n&lt;/em&gt; = 996 patients, 17.2% vs 21.1%, MD = 3.9%, 95% CI: −1.1% to 8.9%).&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Conclusion&lt;/h3&gt;&lt;div&gt;In this randomized trial, a prize draw incentive modestly improved response rates to a patient-reported outcomes survey in routine emergency care settings. Prosocial framing may have slightly increased response rates, but the effect was uncertain. Both behavioral approaches warrant further testing in routine care settings.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Plain Language Summary&lt;/h3&gt;&lt;div&gt;Patient-reported outcomes, such as surveys about how people feel and recover after care, are important for understanding what matters most to patients. However, response rates to these surveys are often very low, especially in real clinical settings. This makes it difficult to draw strong conclusions about whether treatments are helping patients. So, researchers and health services need to find ways to improve response rates. This study looked at whether simple text message strategies could encourage more patients to com","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"191 ","pages":"Article 112116"},"PeriodicalIF":5.2,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145806204","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Paper 1: a semi-automated approach facilitated the assessment of the certainty of evidence in a network meta-analysis: part 1 – Direct comparisons","authors":"Mohammed Mujaab Kamso ,&nbsp;Samuel L. Whittle ,&nbsp;Jordi Pardo Pardo ,&nbsp;Rachelle Buchbinder ,&nbsp;George Wells ,&nbsp;Rob Deardon ,&nbsp;Tolulope Sajobi ,&nbsp;George Tomlinson ,&nbsp;Jesse Elliott ,&nbsp;Jocelyn Thomas ,&nbsp;Shannon E. Kelly ,&nbsp;Romina Brignardello-Petersen ,&nbsp;Glen S. Hazlewood","doi":"10.1016/j.jclinepi.2025.112109","DOIUrl":"10.1016/j.jclinepi.2025.112109","url":null,"abstract":"<div><h3>Objectives</h3><div>To implement and evaluate a semi-automated approach to facilitate rating the Grading, Recommendation, Assessment, Development and Evaluation (GRADE) certainty of evidence (CoE) for direct comparisons within two living network meta-analysis.</div></div><div><h3>Methods</h3><div>For each of three GRADE domains (study limitations, indirectness, and inconsistency), decision rules were developed and used to generate automated judgments for each domain and the overall certainty. Inputs included risk of bias and indirectness ratings for each study and measures of heterogeneity. Indirectness ratings were made by two independent reviewers and resolved through consensus. With the help of an online tool (customized to our project), two independent raters viewed forest plots and additional data and could confirm or modify the suggested rating. Disagreements were resolved by consensus. We evaluated inter-rater reliability and accuracy.</div></div><div><h3>Results</h3><div>Across 374 direct comparisons, there was perfect agreement (100%) between the automated judgment and reviewer consensus, when only a single study was available (<em>n</em> = 292), and near-perfect agreement when more than one study was available (99%–100% for the three GRADE domains and 96% for overall rating). Inter-rater reliability was near perfect (Gwet's AC1 kappa score ranging from 96% to 100%).</div></div><div><h3>Conclusion</h3><div>Automated judgments using established decision rules agreed with expert judgment for the vast majority of GRADE CoE ratings.</div></div>","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"191 ","pages":"Article 112109"},"PeriodicalIF":5.2,"publicationDate":"2025-12-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145800878","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The opacity and exemption of artificial intelligence or the epic of explainable artificial intelligence, reply to commentary by Rattanapitoon et al","authors":"Manuel Marques-Cruz,&nbsp;Rafael José Vieira,&nbsp;Sara Gil Mata,&nbsp;Bernardo Sousa-Pinto","doi":"10.1016/j.jclinepi.2025.112111","DOIUrl":"10.1016/j.jclinepi.2025.112111","url":null,"abstract":"","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"191 ","pages":"Article 112111"},"PeriodicalIF":5.2,"publicationDate":"2025-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145795605","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Health experiences and inequalities across intersecting social identities in health research: a scoping review.","authors":"Azar Alexander-Sefre, Frances Sherratt, Heidi Green, Shaun Treweek, Victoria Shepherd","doi":"10.1016/j.jclinepi.2025.112112","DOIUrl":"https://doi.org/10.1016/j.jclinepi.2025.112112","url":null,"abstract":"<p><strong>Background: </strong>Intersectionality provides a framework to help enable critical thinking about how sociodemographic factors interact. There is currently limited evidence on whether having multiple sociodemographic factors, typically associated with underrepresentation in research (e.g., minority ethnicity, lower socioeconomic status), affect health conditions and outcomes. Given the essential role that clinical trials have in the development of effective treatments, this makes it challenging to address whether intersectionality should be considered in trials.</p><p><strong>Aim/objectives: </strong>This scoping review aimed to map the existing literature on the impact of intersectionality on health inequalities and outcomes in developed economies, and identify how intersecting sociodemographic factors affect health.</p><p><strong>Methods: </strong>Following the Arksey and O'Malley Framework and Joanna Briggs Institute methodology, the review adhered to PRISMA-ScR guidelines. Databases searched included Medline, Embase, Web of Science, International Bibliography of the Social Sciences, and Sociological Abstracts. Selection criteria were based on the Population-Concept-Context mnemonic, targeting studies that explicitly referenced intersecting sociodemographic factors and their impact on health experiences. Data were extracted from the Discussions section of the included studies, specifically any reports of the effects of intersectional sociodemographic factors, such as ethnicity, sex, gender, and socioeconomic status, on health conditions and outcomes.</p><p><strong>Results: </strong>Thirty-three studies met the inclusion criteria. The review found that people who belong to more than one sociodemographic group typically under-served in research (e.g., minoritised ethnic and experience of socioeconomic disadvantage), tend to have poorer health. This review also found that context is an important component, with some traditionally privileged groups (e.g., white, male and with a high socioeconomic background) having relatively poorer health outcomes depending on the context.</p><p><strong>Conclusion: </strong>Overall, possessing greater intersectionality is likely to lead to poorer health, however there is no simple relationship, and context plays a role. These findings emphasise the need for inclusive clinical trials that account for multiple sociodemographic factors and the necessity of designing inclusive research that reflects diverse populations.</p>","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":" ","pages":"112112"},"PeriodicalIF":5.2,"publicationDate":"2025-12-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145783582","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Beautiful weights, misinterpreted effects: the use and misuse of overlap weighting in major medical journals, 2020–2025","authors":"John G. Rizk ,&nbsp;Giuseppe Lippi ,&nbsp;Carl J. Lavie","doi":"10.1016/j.jclinepi.2025.112113","DOIUrl":"10.1016/j.jclinepi.2025.112113","url":null,"abstract":"<div><h3>Objectives</h3><div>To evaluate the implementation and reporting practices of overlap weighting in major medical journals.</div></div><div><h3>Study Design and Setting</h3><div>We reviewed observational studies published from January 2020 to September 2025 in five major medical journals (<em>Annals of Internal Medicine</em>, <em>The British Medical Journal [</em><em>BMJ</em><em>]</em>, <em>Journal of the American Medical Association</em> [<em>JAMA</em>], <em>JAMA Internal Medicine</em>, and <em>The New England Journal of Medicine</em> [<em>NEJM</em>]) that used overlap weighting as a primary or sensitivity adjustment method. Reporting quality was assessed for estimand specification, definition of the overlap population, justification of the method, acknowledgment of advantages, and discussion of interpretability.</div></div><div><h3>Results</h3><div>Seventeen eligible studies were identified. Four studies (24%) correctly named the estimand as the average treatment effect in the overlap population; two (12%) misreported the estimand as average treatment effect, and the remainder did not specify an estimand. Ten studies (59%) described the overlap population at least partially. Sixteen studies (94%) highlighted at least one statistical advantage of overlap weighting, yet none acknowledged that results apply only to the overlap population. These results point to a notable gap in estimand reporting.</div></div><div><h3>Conclusion</h3><div>Clearer specification of the estimand and its target population is essential to prevent misinterpretation. Strengthening reporting standards will support more transparent and appropriate use of overlap weighting in medical research.</div></div><div><h3>Plain Language Summary</h3><div>This study examined how major medical journals (from 2020–2025) report studies using overlap weighting, a method that focuses on patients who could receive either treatment. Most studies acknowledged advantages of using overlap weighting but did not clearly state that results apply only to this “overlap” group. Clear reporting of the target population and effect estimate is needed to avoid misleading interpretations.</div></div>","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"191 ","pages":"Article 112113"},"PeriodicalIF":5.2,"publicationDate":"2025-12-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145783628","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Paper 2: a semi-automated approach facilitated the assessment of the certainty of evidence for in a network meta-analysis: part 2 – indirect and mixed comparisons","authors":"Mohammed Mujaab Kamso ,&nbsp;Samuel L. Whittle ,&nbsp;Jordi Pardo Pardo ,&nbsp;Rachelle Buchbinder ,&nbsp;George Wells ,&nbsp;Rob Deardon ,&nbsp;Tolulope Sajobi ,&nbsp;George Tomlinson ,&nbsp;Jesse Elliott ,&nbsp;Jocelyn Thomas ,&nbsp;Shannon E. Kelly ,&nbsp;Romina Brignardello-Petersen ,&nbsp;Glen S. Hazlewood","doi":"10.1016/j.jclinepi.2025.112110","DOIUrl":"10.1016/j.jclinepi.2025.112110","url":null,"abstract":"<div><h3>Objectives</h3><div>To implement a semiautomated approach to facilitate rating the Grading, Recommendation, Assessment, Development and Evaluation certainty of evidence (CoE) for indirect and network meta-analysis (NMA) estimates.</div></div><div><h3>Methods</h3><div>We developed and implemented algorithms for generating automated ratings for the CoE for indirect and network estimates in two living NMAs of rheumatoid arthritis treatment. At the indirect stage, inputs included CoE ratings for direct estimates and the contribution matrix. Intransitivity ratings were assigned based on the indirectness ratings of the two direct estimates with the highest percent contribution. An online tool (customized to our project) facilitated assessment of imprecision on the network estimate. Automated ratings were reviewed by two independent experts.</div></div><div><h3>Results</h3><div>Across 1306 indirect comparisons, the contribution matrix identified the dominant branches of evidence regardless of whether a single first order loop was present (80%) or not. The reviewers agreed with all automated CoE ratings for incoherence (<em>n</em> = 34), network estimates (<em>n</em> = 34) and imprecision (<em>n</em> = 1447). They agreed with the automated intransitivity algorithm except when the total contribution of the top-two direct estimates was low (eg, &lt;50%, which occurred in 38% of the estimates).</div></div><div><h3>Conclusion</h3><div>Automated approaches facilitated CoE ratings for indirect and network estimates. Further work is required to define appropriate algorithms for intransitivity.</div></div>","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"191 ","pages":"Article 112110"},"PeriodicalIF":5.2,"publicationDate":"2025-12-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145783561","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Comparison of AI-assisted and human-generated plain language summaries for Cochrane reviews: a randomised non-inferiority trial (HIET-1) [Registered Report - stage II]","authors":"Declan Devane ,&nbsp;Johanna Pope ,&nbsp;Paula Byrne ,&nbsp;Evan Forde ,&nbsp;Isabel O'Byrne ,&nbsp;Steven Woloshin ,&nbsp;Eileen Culloty ,&nbsp;Darren Dahly ,&nbsp;Ingeborg Hess Elgersma ,&nbsp;Heather Munthe-Kaas ,&nbsp;Conor Judge ,&nbsp;Martin O'Donnell ,&nbsp;Finn Krewer ,&nbsp;Sandra Galvin ,&nbsp;Nikita N. Burke ,&nbsp;Theresa Tierney ,&nbsp;KM Saif-Ur-Rahman ,&nbsp;Tom Conway ,&nbsp;James Thomas","doi":"10.1016/j.jclinepi.2025.112102","DOIUrl":"10.1016/j.jclinepi.2025.112102","url":null,"abstract":"&lt;div&gt;&lt;h3&gt;Objectives&lt;/h3&gt;&lt;div&gt;To compare the comprehension, readability, quality, safety, and trustworthiness of artificial intelligence (AI)-assisted vs human-generated plain language summaries (PLSs) for Cochrane systematic reviews.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Study Design&lt;/h3&gt;&lt;div&gt;Randomized, parallel-group, two-arm, noninferiority trial (ISRCTN85699985).&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Setting&lt;/h3&gt;&lt;div&gt;Online survey platform, September 2025.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Participants&lt;/h3&gt;&lt;div&gt;Adults aged 18 years or older with a minimum English reading proficiency of 7 out of 10, recruited via Prolific. Of the 500 individuals screened, 465 were randomized and 453 completed per-protocol analysis.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Interventions&lt;/h3&gt;&lt;div&gt;Participants were randomly assigned to three AI-assisted PLSs developed with ChatGPT and human-in-the-loop verification, or to three published human-generated Cochrane PLSs for the same reviews.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Outcomes&lt;/h3&gt;&lt;div&gt;Primary: comprehension (10-item questionnaire, noninferiority margin 10%). Secondary: readability quality and safety, trustworthiness, and authorship perception.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Results&lt;/h3&gt;&lt;div&gt;Mean comprehension scores were 88.9% (&lt;em&gt;n&lt;/em&gt; = 228) in the AI-assisted group and 89.0% (&lt;em&gt;n&lt;/em&gt; = 225) in the human-generated group (mean difference −0.03 percentage points, 95% CI: −1.9% to 2.0%); the upper CI bound (2.0 percentage points) did not exceed the +10 percentage-point noninferiority margin, demonstrating noninferiority. Flesch-Kincaid Grade Level showed no significant difference (8.20 vs 8.38, &lt;em&gt;P&lt;/em&gt; = .722), although formal noninferiority was missed (upper 95% CI bound 1.72 exceeded the 1.0 grade level margin). AI-assisted summaries scored higher on Flesch Reading Ease (63.33 vs 50.00, &lt;em&gt;P&lt;/em&gt; = .008) and lower on the Coleman-Liau Index. All summaries met prespecified quality and safety standards (100% in both groups). Trustworthiness scores were comparable (3.98 vs 3.91, difference 0.068, 95% CI: −0.043 to 0.179; meeting noninferiority). Participants demonstrated limited ability to distinguish between authorship, correctly identifying AI-assisted summaries in 56.3% of cases and human-generated summaries in 34.7% (≈ chance for a three-option question), with 55.4% of human-generated summaries misattributed as AI-assisted. Exploratory subgroup analysis showed an age interaction (&lt;em&gt;P&lt;/em&gt; = .023), though based on a small subgroup (&lt;em&gt;n&lt;/em&gt; = 14, 3%).&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Conclusion&lt;/h3&gt;&lt;div&gt;AI-assisted PLSs with human oversight achieved comprehension levels noninferior to those of human-generated Cochrane summaries, with comparable quality, safety, and trust ratings. AI summaries were largely indistinguishable from those generated by humans. Pretrial verification identified and corrected numerical errors, confirming the need for human oversight. These findings support human-in-the-loop AI workflows for PLS production, though formal evaluation of the time and resource implications is needed","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"191 ","pages":"Article 112102"},"PeriodicalIF":5.2,"publicationDate":"2025-12-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145897844","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}