Journal of Clinical Epidemiology最新文献

{"title":"Potential waste in evidence synthesis for health screening: a scoping review and call for action","authors":"Sarah Batson ,&nbsp;Matthew J. Randell ,&nbsp;Catherine Bane ,&nbsp;Julia Geppert ,&nbsp;Pranshu Mundada ,&nbsp;Chris Stinton ,&nbsp;Eleanor Cozens ,&nbsp;Maggie Powell ,&nbsp;Sian Taylor-Phillips","doi":"10.1016/j.jclinepi.2025.112053","DOIUrl":"10.1016/j.jclinepi.2025.112053","url":null,"abstract":"<div><h3>Objectives</h3><div>To estimate the extent of potential research waste in the production of evidence-synthesis products for health screening, providing an initial assessment of its magnitude.</div></div><div><h3>Study Design and Setting</h3><div>Evidence-synthesis products supporting screening recommendations for adult populations, published by the UK National Screening Committee (UK NSC) and the US Preventive Services Task Force (USPSTF) between 2014 and 2024, were identified as anchor reviews. For each anchor review, Embase, Medline, and national and international organization websites were searched for overlapping evidence reviews on the same topic, defined as addressing the same research questions with at least partial overlap in the population, interventions, comparisons, and outcomes.</div></div><div><h3>Results</h3><div>A total of 48 anchor reviews (covering 33 conditions) were identified from the UK NSC and USPSTF. Overlapping evidence reviews were identified for 92% (44/48) of these, with a median of 4 additional reviews per anchor review (range: 0–60; interquartile range [IQR]: 2–15). Of the overlapping reviews, 11% showed overlap with partial use of prior work, explicitly updating or building upon prior external work, but with new elements and scope differences that maintained their classification as overlapping. Focusing on a core subset of conditions of shared interest to both organizations, the median overlap increased to 13 (range: 2–47; IQR: 4–17), indicating substantial duplication in priority areas. Seventy percent of all reviews in the evidence base were conducted in North America (28%) and Western Europe (42%), with limited representation from low- and middle-income countries.</div></div><div><h3>Conclusion</h3><div>The results of this review highlight potential research waste due to duplication in evidence synthesis efforts. Coordinated action among organizations advising policymakers, such as NSCs, public health agencies, and evidence review bodies may help establish more efficient, collaborative approaches that enable reuse and adaptation across contexts. Such action could include real-time sharing of ongoing reviews, multiregion comprehensive reviews, and the use of stratified analyses to tailor findings to country-specific needs. These strategies should be explored to determine whether organizations can reduce unnecessary duplication, enhance equity, improve the timeliness and relevance of guidance, and redirect resources toward unmet research priorities and other pressing public health challenges.</div></div>","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"190 ","pages":"Article 112053"},"PeriodicalIF":5.2,"publicationDate":"2025-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145558295","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Developing a framework for assessing the applicability of the target condition in diagnostic research","authors":"Eve Tomlinson ,&nbsp;Jude Holmes ,&nbsp;Anne W.S. Rutjes ,&nbsp;Clare Davenport ,&nbsp;Mariska Leeflang ,&nbsp;Bada Yang ,&nbsp;Sue Mallett ,&nbsp;Penny Whiting","doi":"10.1016/j.jclinepi.2025.112059","DOIUrl":"10.1016/j.jclinepi.2025.112059","url":null,"abstract":"&lt;div&gt;&lt;h3&gt;Objectives&lt;/h3&gt;&lt;div&gt;Assessment of the applicability of primary studies is an essential but often a challenging aspect of systematic reviews of diagnostic test accuracy studies (DTA reviews). We explored review authors’ applicability assessments for the QUADAS-2 reference standard domain within Cochrane DTA reviews. We highlight applicability concerns, identify potential issues with assessment, and develop a framework for assessing the applicability of the target condition as defined by the reference standard.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Study Design and Setting&lt;/h3&gt;&lt;div&gt;Methodological review. DTA reviews in the Cochrane Library that used QUADAS-2 and judged applicability for the reference standard domain as “high concern” for at least one study were eligible. One reviewer extracted the rationale for the “high concern” and this was checked by a second reviewer. Two reviewers categorized the rationale inductively into themes, and a third reviewer verified these. Discussions regarding the extracted information informed framework development.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Results&lt;/h3&gt;&lt;div&gt;We identified 50 eligible reviews. Five themes emerged: study uses different reference standard threshold to define the target condition (six reviews), misclassification by the reference standard in the study such that the target condition in the study does not match the review question (11 reviews), reference standard could not be applied to all participants resulting in a different target condition (five reviews), misunderstanding QUADAS-2 applicability (seven reviews), and insufficient information (21 reviews). Our framework for researchers outlines four potential applicability concerns for the assessment of the target condition as defined by the reference standard: different sub-categories of the target condition, different threshold used to define the target condition, reference standard not applied to full study group, and misclassification of the target condition by the reference standard.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Conclusion&lt;/h3&gt;&lt;div&gt;Clear sources of applicability concerns are identifiable, but several Cochrane review authors struggle to adequately identify and report them. We have developed an applicability framework to guide review authors in their assessment of applicability concerns for the QUADAS reference standard domain.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Plain Language Summary&lt;/h3&gt;&lt;div&gt;What is the problem? Doctors use tests to help to decide if a person has a certain condition. They want to know how accurate the test is before they use it. This means how well it can tell people who have the condition from people who do not have it. This information can be found in “diagnostic systematic reviews”. Diagnostic systematic reviews start with a research question. They bring together findings from studies that have already been done to try to answer this question. It is important for researchers to check that the studies match the review question. This is called an “applicability assess","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"190 ","pages":"Article 112059"},"PeriodicalIF":5.2,"publicationDate":"2025-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145543777","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Challenges in using GRADE by systematic review authors and how to overcome them: a response to Andric et al.","authors":"Miranda W. Langendam,&nbsp;Ignacio Neumann,&nbsp;Holger J. Schünemann","doi":"10.1016/j.jclinepi.2025.112055","DOIUrl":"10.1016/j.jclinepi.2025.112055","url":null,"abstract":"","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"190 ","pages":"Article 112055"},"PeriodicalIF":5.2,"publicationDate":"2025-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145543732","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Estimands: what they are and why we should use them","authors":"Brennan C. Kahan ,&nbsp;Declan Devane","doi":"10.1016/j.jclinepi.2025.112054","DOIUrl":"10.1016/j.jclinepi.2025.112054","url":null,"abstract":"<div><div>In clinical trials, postrandomization events, such as treatment discontinuation or the use of rescue medication, can complicate the interpretation of results. An estimand is a precise description of the treatment effect that investigators wish to estimate. Estimands facilitate more straightforward interpretation of trial results by explicitly defining how postrandomization “intercurrent” events are incorporated into the research question. This article introduces the five key attributes of estimands (population, treatment conditions, endpoint, summary measure, and strategies for intercurrent events) and explains the five main strategies for managing intercurrent events (treatment policy, composite, while on treatment, hypothetical, and principal stratum). Using a practical example of a trial comparing cognitive behavioral therapy vs medication for mild anxiety, we demonstrate how different estimand choices lead to varying study designs, analyses, and interpretations. Understanding estimands helps researchers design better trials and enables stakeholders to determine if the results are relevant to their situation. We also explain how sensitivity analyses can be used to check the reliability of results by assessing how results change under different statistical assumptions.</div></div>","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"189 ","pages":"Article 112054"},"PeriodicalIF":5.2,"publicationDate":"2025-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145543764","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Identifying ethical issues in stepped-wedge cluster randomized trials to inform the Ottawa Statement update: a systematic review of trials published 2016–2022","authors":"Cory E. Goldstein ,&nbsp;Julia F. Shaw ,&nbsp;Ariella Binik ,&nbsp;Hayden P. Nix ,&nbsp;Luis Ortiz-Reyes ,&nbsp;Thais Mazzetti ,&nbsp;Anand Sergeant ,&nbsp;Charles Weijer ,&nbsp;Monica Taljaard","doi":"10.1016/j.jclinepi.2025.112051","DOIUrl":"10.1016/j.jclinepi.2025.112051","url":null,"abstract":"&lt;div&gt;&lt;h3&gt;Objectives&lt;/h3&gt;&lt;div&gt;The &lt;em&gt;Ottawa Statement on the Ethical Design and Conduct of Cluster Randomized Trials&lt;/em&gt; (CRTs), published in 2012, is the first and remains the only international ethics guidance document specific to CRTs. However, the stepped-wedge CRT design raises complex ethical issues for which guidance may be lacking. The overarching objective of this review is to inform the forthcoming update of the &lt;em&gt;Ottawa Statement&lt;/em&gt;; specific objectives are to characterize the types of interventions and data collection procedures in stepped-wedge CRTs (SW-CRTs), and to examine adherence to key ethical design, conduct, and reporting recommendations.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Study Design and Setting&lt;/h3&gt;&lt;div&gt;Primary reports of SW-CRTs evaluating health interventions published 2016-2022 in English were reviewed. Two reviewers extracted data from each trial independently; discrepancies were resolved through consensus.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Results&lt;/h3&gt;&lt;div&gt;Among 160 SW-CRTs, most evaluated multilevel interventions (78, 49%), and uncommonly involved therapeutic patient interventions (19, 12%). Few (10, 6%) exclusively used routinely collected data sources for outcome assessment. Sixty-four trials (40%) provided explicit justifications for using cluster randomization and the stepped-wedge design. Most (157, 98%) included a statement about research ethics committee (REC) review, of which 148 (94%) reported approval. A statement about consent was reported in 145 (91%), with 113 (78%) pertaining to patients only, 10 (7%) to health-care providers only, and 22 (15%) to both. Among 135 trials reporting on patient consent, consent was not obtained in 55 (41%). Justifications for not obtaining consent were provided in 42 (76%).&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Conclusion&lt;/h3&gt;&lt;div&gt;The updated &lt;em&gt;Ottawa Statement&lt;/em&gt; should provide guidance about when people should be considered research participants and when their consent is required, justifications for using a stepped-wedge design, the need for REC review, and the burden of data collection procedures in SW-CRTs.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Plain Language Summary&lt;/h3&gt;&lt;div&gt;We reviewed 160 SW-CRTs published between 2016 and 2022. These trials often evaluate multifaceted interventions delivered at the cluster or professional level rather than individual level, and most had to rely on at least some form of primary data collection. Although almost all SW-CRTs reported ethics review, fewer than half explained why they used the SW-CRT design, and a substantial minority did not include a statement about whether participant informed consent was obtained. Rarely did consent statement pertain to health-care providers; most statements pertained to patients. Among the SW-CRTs that had consent statements pertaining to patients, almost half reported not obtaining or waiving consent from them, and many of these did not report clear reasons for this. Our findings highlight the need for clearer guidance in the forthcoming update to the &lt;e","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"189 ","pages":"Article 112051"},"PeriodicalIF":5.2,"publicationDate":"2025-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145530801","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Scalable medication extraction and discontinuation identification from electronic health records using large language models","authors":"Chong Shao ,&nbsp;Douglas Snyder ,&nbsp;Chiran Li ,&nbsp;Bowen Gu ,&nbsp;Kerry Ngan ,&nbsp;Chun-Ting Yang ,&nbsp;Jiageng Wu ,&nbsp;Richard Wyss ,&nbsp;Kueiyu Joshua Lin ,&nbsp;Jie Yang","doi":"10.1016/j.jclinepi.2025.112049","DOIUrl":"10.1016/j.jclinepi.2025.112049","url":null,"abstract":"&lt;div&gt;&lt;h3&gt;Objectives&lt;/h3&gt;&lt;div&gt;Identifying medication discontinuations in electronic health records (EHRs) is vital for patient safety but is often hindered by information being buried in unstructured notes. This study aims to evaluate the capabilities of advanced open-sourced and proprietary large language models in extracting medications and classifying their medication status from EHR notes, focusing on their scalability for medication information extraction without human annotation.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Study Design and Setting&lt;/h3&gt;&lt;div&gt;We collected three EHR datasets from diverse sources to build the evaluation benchmark: 1 publicly available dataset (Reannotated Clinical Acronym Sense Inventory dataset [Re-CASI]), 1 we annotated based on public MIMIC notes (MIMIC-IV Medication Snippet dataset [MIV-Med]), and 1 internally annotated on clinical notes from Mass General Brigham (MGB-Med). We evaluated 12 advanced LLMs, including general-domain open-sourced models (eg, Llama-3.1-70B-Instruct, Qwen2.5-72B-Instruct), medical-specific models (eg, MeLLaMA-70B-chat), and a proprietary model (GPT-4o). We explored multiple LLM prompting strategies, including zero-shot, 5-shot, and Chain-of-Thought (CoT) approaches. Performance on medication extraction, medication status classification, and their joint task (extraction then classification) was systematically compared across all experiments.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Results&lt;/h3&gt;&lt;div&gt;LLMs showed promising performance on medication extraction, while discontinuation classification and joint tasks were more challenging. GPT-4o consistently achieved the highest average F1 scores in all tasks under zero-shot setting — 94.0% for medication extraction, 78.1% for discontinuation classification, and 72.7% for the joint task. Open-sourced models followed closely, with Llama-3.1-70B-Instruct achieving the highest performance in medication status classification on the MIV-Med dataset (68.7%) and in the joint task on both the Re-CASI (76.2%) and MIV-Med (60.2%) datasets. Medical-specific LLMs demonstrated lower performance compared to advanced general-domain LLMs. Few-shot learning generally improved performance, while CoT reasoning showed inconsistent gains. Notably, open-sourced models occasionally surpassed GPT-4o performance, underscoring their potential in privacy-sensitive clinical research.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Conclusion&lt;/h3&gt;&lt;div&gt;LLMs demonstrate strong potential for medication extraction and discontinuation identification on EHR notes, with open-sourced models offering scalable alternatives to proprietary systems and few-shot learning further improving LLMs’ capability.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Plain Language Summary&lt;/h3&gt;&lt;div&gt;Stopping a medicine can affect safety and treatment decisions, yet this detail is often buried in long electronic health record notes. We evaluated whether large language models, which read and summarize text, can automatically find medication names and decide whether each medicine is still being take","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"189 ","pages":"Article 112049"},"PeriodicalIF":5.2,"publicationDate":"2025-11-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145514496","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Dysinclusion: naming and defining the inequitable absence of marginalized populations in health research","authors":"Anna Durbin ,&nbsp;Lisa Whittingham ,&nbsp;Anjali Menezes ,&nbsp;Lucie Richard ,&nbsp;Janet Durbin ,&nbsp;Aaron M. Orkin","doi":"10.1016/j.jclinepi.2025.112048","DOIUrl":"10.1016/j.jclinepi.2025.112048","url":null,"abstract":"&lt;div&gt;&lt;h3&gt;Background and Objectives&lt;/h3&gt;&lt;div&gt;Marginalized populations are frequently absent or invisible in health research. Yet this problem is seldom characterized as a distinct methodological concern. Existing concepts like selection bias or generalizability approach these inequities primarily as technical limitations, not as methodological deficiencies. We introduce &lt;em&gt;dysinclusion&lt;/em&gt; to name and define the inequitable absence or invisibility of groups who should be included in research. Our objective is to establish dysinclusion as a distinct concept at the intersection of equity and methods, distinguish it from existing methodological concepts, and examine how it functions, why it matters, and how it can be addressed.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Methods&lt;/h3&gt;&lt;div&gt;We draw on examples to define dysinclusion and describe its mechanisms. We differentiate dysinclusion from adjacent epidemiological concepts, and propose three types of dysinclusion processes: data coverage, nonparticipation, and invisibility.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Results&lt;/h3&gt;&lt;div&gt;Dysinclusion reveals how structural marginalization becomes embedded in research methods. It occurs when marginalized groups are absent from data sources, excluded through study design or barriers to participation, or rendered invisible by measurement and reporting practices. These patterned absences compromise the validity, relevance, and ethical foundation of research. We argue that dysinclusion should be identified and managed not only as a source of bias or threat to validity, but as a central criterion of methodological rigor in the design and implementation of health research.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Conclusion&lt;/h3&gt;&lt;div&gt;Naming dysinclusion challenges the normalization of exclusion and inequity in health research. Dysinclusion offers language to link the ethical concept of equity with research methods. Making dysinclusion visible reframes patterned absence as a threat to both equity and scientific rigor—one that demands deliberate recognition, accountability, and change.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Plain Language Summary&lt;/h3&gt;&lt;div&gt;Some groups—like people with disabilities, racialized communities, or those living in poverty—are often missing from health research. Even when they face some of the greatest health challenges, these groups are frequently left out of studies, underrepresented in data, or not even recognized as distinct populations. This absence has serious consequences: it limits what we know about their health, weakens the accuracy of research findings, and can make existing health disparities worse. This problem is common, but there is no widely used method or term in health research to describe or address it. Researchers typically think about who is missing from studies in terms of technical issues like bias or generalizability. These concepts do not fully capture the deeper problem of structural inequality, and make it seem as though ethical concerns, like health equity, are separate from the methods that l","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"189 ","pages":"Article 112048"},"PeriodicalIF":5.2,"publicationDate":"2025-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145507665","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Behavior of prediction performance metrics with rare events","authors":"Emily Minus ,&nbsp;R.Yates Coley ,&nbsp;Susan M. Shortreed ,&nbsp;Brian D. Williamson","doi":"10.1016/j.jclinepi.2025.112046","DOIUrl":"10.1016/j.jclinepi.2025.112046","url":null,"abstract":"<div><h3>Objective</h3><div>Area under the receiver operating characteristic curve (AUC) is commonly reported alongside prediction models for binary outcomes. Recent articles have raised concerns that AUC might be a misleading measure of prediction performance in the rare event setting. This setting is common since many events of clinical importance are rare. We aimed to determine whether the bias and variance of AUC are driven by the number of events or the event rate. We also investigated the behavior of other commonly used measures of prediction performance, including positive predictive value, accuracy, sensitivity, and specificity.</div></div><div><h3>Study Design and Setting</h3><div>We conducted a simulation study to determine when or whether AUC is unstable in the rare event setting by varying the size of datasets used to train and evaluate prediction models. This plasmode simulation study was based on data from the Mental Health Research Network; the data contained 149 predictors and the outcome of interest, suicide attempt, which had event rate 0.92% in the original dataset.</div></div><div><h3>Results</h3><div>Our results indicate that poor AUC behavior—as measured by empirical bias, variability of cross-validated AUC estimates, and empirical coverage of confidence intervals—is driven by the number of events in a rare-event setting, not event rate. Performance of sensitivity is driven by the number of events, while that of specificity is driven by the number of nonevents. Other measures, including positive predictive value and accuracy, depend on the event rate even in large samples.</div></div><div><h3>Conclusion</h3><div>AUC is reliable in the rare event setting provided that the total number of events is moderately large; in our simulations, we observed near zero bias with 1000 events.</div></div><div><h3>Plain Language Summary</h3><div>Predicting self-harm or suicidal behavior is medically important for guiding clinicians in providing care to patients. Several research teams have developed and evaluated suicide risk prediction models based on health records data. Part of evaluating these models is calculating area under the receiver operating characteristic curve (AUC) and other prediction performance metrics. Self-harm and suicide are rare events. Recent research has raised concerns with using AUC in rare-event settings. We aimed to determine whether having a sufficiently large dataset could remove these concerns. In our experiments, we found that AUC can be used without concern in settings with 1000 events or more. Thus, AUC is a valid measure of suicide risk prediction model performance in many large healthcare databases.</div></div>","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"189 ","pages":"Article 112046"},"PeriodicalIF":5.2,"publicationDate":"2025-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145507618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Influence of focusing on dominant first order loop when assessing the certainty of evidence of network meta-analysis: a case study","authors":"Liang Yao ,&nbsp;Xu Hui ,&nbsp;Liujiao Cao ,&nbsp;Meixuan Li ,&nbsp;Jinhui Tian ,&nbsp;Yizhuo Chen ,&nbsp;Peijing Yan ,&nbsp;Qi Wang ,&nbsp;Xiaoqin Wang ,&nbsp;Kehu Yang ,&nbsp;Gordon H. Guyatt ,&nbsp;Romina Brignardello-Petersen","doi":"10.1016/j.jclinepi.2025.112050","DOIUrl":"10.1016/j.jclinepi.2025.112050","url":null,"abstract":"<div><h3>Objectives</h3><div>Assessing the certainty of evidence (COE) of both direct and indirect evidence of is crucial for enhancing the understanding of network meta-analysis (NMA) findings and drawing appropriate conclusions. To make the certainty rating of indirect evidence feasible, Grading of Recommendations Assessment, Development, and Evaluation (GRADE) guidance suggests focusing on the dominant first order loop that typically contributes the most information to the indirect estimate. This approach, however, can raise concerns that failure to consider most or all loops will provide inaccurate COE of indirect evidence.</div></div><div><h3>Study Design and Setting</h3><div>We investigated six NMA publications and compared indirect COE ratings considering only the dominant first order loop vs ratings considering most or all loops.</div></div><div><h3>Results</h3><div>Across 103 indirect comparisons in the six NMAs, 15% (15 out of 103) comparisons did not have a first order loop, and 47% (42 out of 88) comparisons with dominant first order loops contributed a weight smaller than 50% of the indirect evidence. We identified a total of 6% (6 out of 103) indirect comparisons in which ratings based on the dominant first order loop resulted in different COE ratings from considering most or all loops, with two shifting from moderate to low COE of NMA.</div></div><div><h3>Conclusion</h3><div>Our findings suggest that using only the dominant first order loop vs considering most or all loops is very seldom misleading. Further research is necessary to replicate or refute our findings in other network meta-analyses and assess the implications for clinical decision-making.</div></div><div><h3>Plain Language Summary</h3><div>NMA allows researchers to compare the relative effectiveness of multiple treatments. The GRADE working group provides guidance to establish the confidence in treatment effects (how likely they are to be true) by evaluating the COE to direct, indirect and NMA estimates, which can be high, moderate, low or very low. We investigated the influences of using two different approaches (dominant first order loop vs consideration of most or all loops) for rating the certainty of NMA. We found similar certainty ratings of indirect evidence and NMA between the two approaches. Therefore, systematic review authors can continue using the simpler approach.</div></div>","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"189 ","pages":"Article 112050"},"PeriodicalIF":5.2,"publicationDate":"2025-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145507670","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Core GRADE unpacked: a summary of recent innovations in complementary GRADE methodology","authors":"Luis Enrique Colunga-Lozano ,&nbsp;Ying Wang ,&nbsp;Thomas Agoritsas ,&nbsp;Monica Hultcrantz ,&nbsp;Alfonso Iorio ,&nbsp;Victor M. Montori ,&nbsp;Gordon Guyatt","doi":"10.1016/j.jclinepi.2025.112047","DOIUrl":"10.1016/j.jclinepi.2025.112047","url":null,"abstract":"&lt;div&gt;&lt;div&gt;The Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework has become the global standard for rating evidence certainty and grading strength of health-care recommendations in systematic reviews, clinical practice guidelines (CPGs), and health technology assessments (HTAs). However, as the methodology has evolved, its growing complexity and difficulties navigating the many articles providing GRADE guidance have created challenges for users. Moreover, GRADE guidance has been published across multiple journals and platforms, resulting in a body of work with considerable redundancy, modifications, and out-of-date material that requires meticulous appraisal of GRADE writings to identify and implement current guidance. In response, a group of GRADE methodologists developed Core GRADE, a simple yet comprehensive framework focused on the essential elements required to apply GRADE to paired comparisons of interventions. Building on prior GRADE publications, the authors reviewed existing guidance and distilled its fundamental components. This proposal represents an independent approach and has not received formal endorsement from the GRADE working group. During this process, they identified multiple areas where clearer recommendations were warranted and incorporated these improvements into the seven Core GRADE papers. This article presents the resulting innovations introduced by core GRADE, that include the use of flow charts and algorithms to guide GRADE implementation; an emphasis on viewing both individual GRADE domains and overall certainty as continua; and clarification of decisions related to addressing potential relative and absolute subgroup effects when formulating patient population, intervention, comparison, and outcome questions.&lt;/div&gt;&lt;/div&gt;&lt;div&gt;&lt;h3&gt;Plain Language Summary&lt;/h3&gt;&lt;div&gt;The GRADE framework was introduced in 2004 to help researchers and health-care professionals assess the certainty (quality) of evidence in systematic reviews and the strength of recommendations in CPGs and HTAs. Over the past 2 decades, it has become overwhelmingly the most widely used approach to making certainty of evidence and strength of recommendation decisions. However, many users now find GRADE increasingly complex, and guidance has appeared in numerous journals and platforms. This has created overlap, outdated information, and inconsistencies that make it difficult to identify the most current best approaches. To address these challenges, GRADE experts developed Core GRADE, a simplified but comprehensive version of GRADE. Core GRADE brings together the essential elements needed to apply GRADE when comparing health-care interventions (1 intervention vs 1 comparator). In addition to this simplification, Core GRADE offers several innovations. These include practical flowcharts and algorithms to guide step-by-step application of the framework; a new emphasis on viewing both individual domains and overall certainty of evidence","PeriodicalId":51079,"journal":{"name":"Journal of Clinical Epidemiology","volume":"189 ","pages":"Article 112047"},"PeriodicalIF":5.2,"publicationDate":"2025-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145482500","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}