Pub Date : 2024-04-16DOI: 10.1186/s41512-024-00170-8
Emmert Roberts, John Strang, Patrick Horgan, Brian Eastwood
{"title":"The development and internal validation of a multivariable model predicting 6-month mortality for people with opioid use disorder presenting to community drug services in England: a protocol","authors":"Emmert Roberts, John Strang, Patrick Horgan, Brian Eastwood","doi":"10.1186/s41512-024-00170-8","DOIUrl":"https://doi.org/10.1186/s41512-024-00170-8","url":null,"abstract":"","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"9 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140698514","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-02DOI: 10.1186/s41512-024-00169-1
Amier Hassan, Brian Critelli, Ila Lahooti, Ali Lahooti, Nate Matzko, Jan Niklas Adams, Lukas Liss, Justin Quion, David Restrepo, Melica Nikahd, Stacey Culp, Lydia Noh, Kathleen Tong, Jun Sung Park, Venkata Akshintala, John A Windsor, Nikhil K Mull, Georgios I Papachristou, Leo Anthony Celi, Peter J Lee
Acute pancreatitis (AP) is an acute inflammatory disorder that is common, costly, and is increasing in incidence worldwide with over 300,000 hospitalizations occurring yearly in the United States alone. As its course and outcomes vary widely, a critical knowledge gap in the field has been a lack of accurate prognostic tools to forecast AP patients' outcomes. Despite several published studies in the last three decades, the predictive performance of published prognostic models has been found to be suboptimal. Recently, non-regression machine learning models (ML) have garnered intense interest in medicine for their potential for better predictive performance. Each year, an increasing number of AP models are being published. However, their methodologic quality relating to transparent reporting and risk of bias in study design has never been systematically appraised. Therefore, through collaboration between a group of clinicians and data scientists with appropriate content expertise, we will perform a systematic review of papers published between January 2021 and December 2023 containing artificial intelligence prognostic models in AP. To systematically assess these studies, the authors will leverage the CHARMS checklist, PROBAST tool for risk of bias assessment, and the most current version of the TRIPOD-AI. (Research Registry ( http://www.reviewregistry1727 .).
{"title":"Critical appraisal of machine learning prognostic models for acute pancreatitis: protocol for a systematic review.","authors":"Amier Hassan, Brian Critelli, Ila Lahooti, Ali Lahooti, Nate Matzko, Jan Niklas Adams, Lukas Liss, Justin Quion, David Restrepo, Melica Nikahd, Stacey Culp, Lydia Noh, Kathleen Tong, Jun Sung Park, Venkata Akshintala, John A Windsor, Nikhil K Mull, Georgios I Papachristou, Leo Anthony Celi, Peter J Lee","doi":"10.1186/s41512-024-00169-1","DOIUrl":"10.1186/s41512-024-00169-1","url":null,"abstract":"<p><p>Acute pancreatitis (AP) is an acute inflammatory disorder that is common, costly, and is increasing in incidence worldwide with over 300,000 hospitalizations occurring yearly in the United States alone. As its course and outcomes vary widely, a critical knowledge gap in the field has been a lack of accurate prognostic tools to forecast AP patients' outcomes. Despite several published studies in the last three decades, the predictive performance of published prognostic models has been found to be suboptimal. Recently, non-regression machine learning models (ML) have garnered intense interest in medicine for their potential for better predictive performance. Each year, an increasing number of AP models are being published. However, their methodologic quality relating to transparent reporting and risk of bias in study design has never been systematically appraised. Therefore, through collaboration between a group of clinicians and data scientists with appropriate content expertise, we will perform a systematic review of papers published between January 2021 and December 2023 containing artificial intelligence prognostic models in AP. To systematically assess these studies, the authors will leverage the CHARMS checklist, PROBAST tool for risk of bias assessment, and the most current version of the TRIPOD-AI. (Research Registry ( http://www.reviewregistry1727 .).</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"8 1","pages":"6"},"PeriodicalIF":0.0,"publicationDate":"2024-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10986113/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140337854","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-19DOI: 10.1186/s41512-024-00168-2
Michael Bonares, Stacey Fisher, Kieran Quinn, Kirsten Wentlandt, Peter Tanuseputro
Background: Patients with dementia and their caregivers could benefit from advance care planning though may not be having these discussions in a timely manner or at all. A prognostic tool could serve as a prompt to healthcare providers to initiate advance care planning among patients and their caregivers, which could increase the receipt of care that is concordant with their goals. Existing prognostic tools have limitations. We seek to develop and validate a clinical prediction tool to estimate the risk of 1-year mortality among hospitalized patients with dementia.
Methods: The derivation cohort will include approximately 235,000 patients with dementia, who were admitted to hospital in Ontario from April 1st, 2009, to December 31st, 2017. Predictor variables will be fully prespecified based on a literature review of etiological studies and existing prognostic tools, and on subject-matter expertise; they will be categorized as follows: sociodemographic factors, comorbidities, previous interventions, functional status, nutritional status, admission information, previous health care utilization. Data-driven selection of predictors will be avoided. Continuous predictors will be modelled as restricted cubic splines. The outcome variable will be mortality within 1 year of admission, which will be modelled as a binary variable, such that a logistic regression model will be estimated. Predictor and outcome variables will be derived from linked population-level healthcare administrative databases. The validation cohort will comprise about 63,000 dementia patients, who were admitted to hospital in Ontario from January 1st, 2018, to March 31st, 2019. Model performance, measured by predictive accuracy, discrimination, and calibration, will be assessed using internal (temporal) validation. Calibration will be evaluated in the total validation cohort and in subgroups of importance to clinicians and policymakers. The final model will be based on the full cohort.
Discussion: We seek to develop and validate a clinical prediction tool to estimate the risk of 1-year mortality among hospitalized patients with dementia. The model would be integrated into the electronic medical records of hospitals to automatically output 1-year mortality risk upon hospitalization. The tool could serve as a trigger for advance care planning and inform access to specialist palliative care services with prognosis-based eligibility criteria. Before implementation, the tool will require external validation and study of its potential impact on clinical decision-making and patient outcomes.
{"title":"Study protocol for the development and validation of a clinical prediction tool to estimate the risk of 1-year mortality among hospitalized patients with dementia.","authors":"Michael Bonares, Stacey Fisher, Kieran Quinn, Kirsten Wentlandt, Peter Tanuseputro","doi":"10.1186/s41512-024-00168-2","DOIUrl":"10.1186/s41512-024-00168-2","url":null,"abstract":"<p><strong>Background: </strong>Patients with dementia and their caregivers could benefit from advance care planning though may not be having these discussions in a timely manner or at all. A prognostic tool could serve as a prompt to healthcare providers to initiate advance care planning among patients and their caregivers, which could increase the receipt of care that is concordant with their goals. Existing prognostic tools have limitations. We seek to develop and validate a clinical prediction tool to estimate the risk of 1-year mortality among hospitalized patients with dementia.</p><p><strong>Methods: </strong>The derivation cohort will include approximately 235,000 patients with dementia, who were admitted to hospital in Ontario from April 1st, 2009, to December 31st, 2017. Predictor variables will be fully prespecified based on a literature review of etiological studies and existing prognostic tools, and on subject-matter expertise; they will be categorized as follows: sociodemographic factors, comorbidities, previous interventions, functional status, nutritional status, admission information, previous health care utilization. Data-driven selection of predictors will be avoided. Continuous predictors will be modelled as restricted cubic splines. The outcome variable will be mortality within 1 year of admission, which will be modelled as a binary variable, such that a logistic regression model will be estimated. Predictor and outcome variables will be derived from linked population-level healthcare administrative databases. The validation cohort will comprise about 63,000 dementia patients, who were admitted to hospital in Ontario from January 1st, 2018, to March 31st, 2019. Model performance, measured by predictive accuracy, discrimination, and calibration, will be assessed using internal (temporal) validation. Calibration will be evaluated in the total validation cohort and in subgroups of importance to clinicians and policymakers. The final model will be based on the full cohort.</p><p><strong>Discussion: </strong>We seek to develop and validate a clinical prediction tool to estimate the risk of 1-year mortality among hospitalized patients with dementia. The model would be integrated into the electronic medical records of hospitals to automatically output 1-year mortality risk upon hospitalization. The tool could serve as a trigger for advance care planning and inform access to specialist palliative care services with prognosis-based eligibility criteria. Before implementation, the tool will require external validation and study of its potential impact on clinical decision-making and patient outcomes.</p><p><strong>Trial registration: </strong>NCT05371782.</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"8 1","pages":"5"},"PeriodicalIF":0.0,"publicationDate":"2024-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10949607/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140159637","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Background: There is urgent clinical need to identify reliable prognostic biomarkers that predict the progression of dementia symptoms in individuals with early-phase Alzheimer's disease (AD) especially given the research on and predicted applications of amyloid-beta (Aβ)-directed immunotherapies to remove Aβ from the brain. Cross-sectional studies have reported higher levels of cerebrospinal fluid and blood glial fibrillary acidic protein (GFAP) in individuals with AD-associated dementia than in cognitively unimpaired individuals. Further, recent longitudinal studies have assessed the prognostic potential of baseline blood GFAP levels as a predictor of future cognitive decline in cognitively unimpaired individuals and in those with mild cognitive impairment (MCI) due to AD. In this systematic review and meta-analysis, we propose analyzing longitudinal studies on blood GFAP levels to predict future cognitive decline.
Methods: This study will include prospective and retrospective cohort studies that assessed blood GFAP levels as a prognostic factor and any prediction models that incorporated blood GFAP levels in cognitively unimpaired individuals or those with MCI. The primary outcome will be conversion to MCI or AD in cognitively unimpaired individuals or conversion to AD in individuals with MCI. Articles from PubMed and Embase will be extracted up to December 31, 2023, without language restrictions. An independent dual screening of abstracts and potentially eligible full-text reports will be conducted. Data will be dual-extracted using the CHeck list for critical appraisal, data extraction for systematic Reviews of prediction Modeling Studies (CHARMS)-prognostic factor, and CHARMS checklists, and we will dual-rate the risk of bias and applicability using the Quality In Prognosis Studies and Prediction Study Risk-of-Bias Assessment tools. We will qualitatively synthesize the study data, participants, index biomarkers, predictive model characteristics, and clinical outcomes. If appropriate, random-effects meta-analyses will be performed to obtain summary estimates. Finally, we will assess the body of evidence using the Grading of Recommendation, Assessment, Development, and Evaluation Approach.
Discussion: This systematic review and meta-analysis will comprehensively evaluate and synthesize existing evidence on blood GFAP levels for prognosticating presymptomatic individuals and those with MCI to help advance risk-stratified treatment strategies for early-phase AD.
{"title":"Blood levels of glial fibrillary acidic protein for predicting clinical progression to Alzheimer's disease in adults without dementia: a systematic review and meta-analysis protocol.","authors":"Takashi Nihashi, Keita Sakurai, Takashi Kato, Yasuyuki Kimura, Kengo Ito, Akinori Nakamura, Teruhiko Terasawa","doi":"10.1186/s41512-024-00167-3","DOIUrl":"10.1186/s41512-024-00167-3","url":null,"abstract":"<p><strong>Background: </strong>There is urgent clinical need to identify reliable prognostic biomarkers that predict the progression of dementia symptoms in individuals with early-phase Alzheimer's disease (AD) especially given the research on and predicted applications of amyloid-beta (Aβ)-directed immunotherapies to remove Aβ from the brain. Cross-sectional studies have reported higher levels of cerebrospinal fluid and blood glial fibrillary acidic protein (GFAP) in individuals with AD-associated dementia than in cognitively unimpaired individuals. Further, recent longitudinal studies have assessed the prognostic potential of baseline blood GFAP levels as a predictor of future cognitive decline in cognitively unimpaired individuals and in those with mild cognitive impairment (MCI) due to AD. In this systematic review and meta-analysis, we propose analyzing longitudinal studies on blood GFAP levels to predict future cognitive decline.</p><p><strong>Methods: </strong>This study will include prospective and retrospective cohort studies that assessed blood GFAP levels as a prognostic factor and any prediction models that incorporated blood GFAP levels in cognitively unimpaired individuals or those with MCI. The primary outcome will be conversion to MCI or AD in cognitively unimpaired individuals or conversion to AD in individuals with MCI. Articles from PubMed and Embase will be extracted up to December 31, 2023, without language restrictions. An independent dual screening of abstracts and potentially eligible full-text reports will be conducted. Data will be dual-extracted using the CHeck list for critical appraisal, data extraction for systematic Reviews of prediction Modeling Studies (CHARMS)-prognostic factor, and CHARMS checklists, and we will dual-rate the risk of bias and applicability using the Quality In Prognosis Studies and Prediction Study Risk-of-Bias Assessment tools. We will qualitatively synthesize the study data, participants, index biomarkers, predictive model characteristics, and clinical outcomes. If appropriate, random-effects meta-analyses will be performed to obtain summary estimates. Finally, we will assess the body of evidence using the Grading of Recommendation, Assessment, Development, and Evaluation Approach.</p><p><strong>Discussion: </strong>This systematic review and meta-analysis will comprehensively evaluate and synthesize existing evidence on blood GFAP levels for prognosticating presymptomatic individuals and those with MCI to help advance risk-stratified treatment strategies for early-phase AD.</p><p><strong>Trial registration: </strong>PROSPERO CRD42023481200.</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"8 1","pages":"4"},"PeriodicalIF":0.0,"publicationDate":"2024-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10913586/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140029676","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-13DOI: 10.1186/s41512-024-00166-4
Alpamys Issanov, Atul Aravindakshan, Lorri Puil, Martin C Tammemägi, Stephen Lam, Trevor J B Dummer
Background: Lung cancer is one of the most commonly diagnosed cancers and the leading cause of cancer-related death worldwide. Although smoking is the primary cause of the cancer, lung cancer is also commonly diagnosed in people who have never smoked. Currently, the proportion of people who have never smoked diagnosed with lung cancer is increasing. Despite this alarming trend, this population is ineligible for lung screening. With the increasing proportion of people who have never smoked among lung cancer cases, there is a pressing need to develop prediction models to identify high-risk people who have never smoked and include them in lung cancer screening programs. Thus, our systematic review is intended to provide a comprehensive summary of the evidence on existing risk prediction models for lung cancer in people who have never smoked.
Methods: Electronic searches will be conducted in MEDLINE (Ovid), Embase (Ovid), Web of Science Core Collection (Clarivate Analytics), Scopus, and Europe PMC and Open-Access Theses and Dissertations databases. Two reviewers will independently perform title and abstract screening, full-text review, and data extraction using the Covidence review platform. Data extraction will be performed based on the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS). The risk of bias will be evaluated independently by two reviewers using the Prediction model Risk-of-Bias Assessment Tool (PROBAST) tool. If a sufficient number of studies are identified to have externally validated the same prediction model, we will combine model performance measures to evaluate the model's average predictive accuracy (e.g., calibration, discrimination) across diverse settings and populations and explore sources of heterogeneity.
Discussion: The results of the review will identify risk prediction models for lung cancer in people who have never smoked. These will be useful for researchers planning to develop novel prediction models, and for clinical practitioners and policy makers seeking guidance for clinical decision-making and the formulation of future lung cancer screening strategies for people who have never smoked.
Systematic review registration: This protocol has been registered in PROSPERO under the registration number CRD42023483824.
背景:肺癌是最常诊断出的癌症之一,也是全球癌症相关死亡的主要原因。虽然吸烟是导致肺癌的主要原因,但从未吸烟的人也常被诊断出肺癌。目前,从未吸烟者被确诊为肺癌的比例正在上升。尽管这一趋势令人担忧,但这部分人群却没有资格接受肺部筛查。随着从未吸烟者在肺癌病例中所占比例的增加,迫切需要开发预测模型来识别从未吸烟的高危人群,并将其纳入肺癌筛查计划。因此,我们的系统性综述旨在对现有的从未吸烟者肺癌风险预测模型的证据进行全面总结:将在MEDLINE(Ovid)、Embase(Ovid)、Web of Science Core Collection(Clarivate Analytics)、Scopus、Europe PMC和Open-Access Theses and Dissertations数据库中进行电子检索。两名审稿人将使用 Covidence 审稿平台独立进行标题和摘要筛选、全文审阅和数据提取。数据提取将根据预测建模研究系统性综述批判性评估和数据提取清单(CHARMS)进行。偏倚风险将由两名审稿人使用预测模型偏倚风险评估工具 (PROBAST) 独立评估。如果发现有足够数量的研究对同一预测模型进行了外部验证,我们将结合模型的性能指标,评估该模型在不同环境和人群中的平均预测准确性(如校准、区分度),并探索异质性的来源:综述结果将确定从未吸烟人群的肺癌风险预测模型。讨论:综述结果将确定从未吸烟人群肺癌风险预测模型,这些模型将对计划开发新型预测模型的研究人员、临床从业人员和政策制定者有所帮助,这些人员和政策制定者正在为从未吸烟人群的临床决策和未来肺癌筛查策略的制定寻求指导:本方案已在 PROSPERO 注册,注册号为 CRD42023483824。
{"title":"Risk prediction models for lung cancer in people who have never smoked: a protocol of a systematic review.","authors":"Alpamys Issanov, Atul Aravindakshan, Lorri Puil, Martin C Tammemägi, Stephen Lam, Trevor J B Dummer","doi":"10.1186/s41512-024-00166-4","DOIUrl":"10.1186/s41512-024-00166-4","url":null,"abstract":"<p><strong>Background: </strong>Lung cancer is one of the most commonly diagnosed cancers and the leading cause of cancer-related death worldwide. Although smoking is the primary cause of the cancer, lung cancer is also commonly diagnosed in people who have never smoked. Currently, the proportion of people who have never smoked diagnosed with lung cancer is increasing. Despite this alarming trend, this population is ineligible for lung screening. With the increasing proportion of people who have never smoked among lung cancer cases, there is a pressing need to develop prediction models to identify high-risk people who have never smoked and include them in lung cancer screening programs. Thus, our systematic review is intended to provide a comprehensive summary of the evidence on existing risk prediction models for lung cancer in people who have never smoked.</p><p><strong>Methods: </strong>Electronic searches will be conducted in MEDLINE (Ovid), Embase (Ovid), Web of Science Core Collection (Clarivate Analytics), Scopus, and Europe PMC and Open-Access Theses and Dissertations databases. Two reviewers will independently perform title and abstract screening, full-text review, and data extraction using the Covidence review platform. Data extraction will be performed based on the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS). The risk of bias will be evaluated independently by two reviewers using the Prediction model Risk-of-Bias Assessment Tool (PROBAST) tool. If a sufficient number of studies are identified to have externally validated the same prediction model, we will combine model performance measures to evaluate the model's average predictive accuracy (e.g., calibration, discrimination) across diverse settings and populations and explore sources of heterogeneity.</p><p><strong>Discussion: </strong>The results of the review will identify risk prediction models for lung cancer in people who have never smoked. These will be useful for researchers planning to develop novel prediction models, and for clinical practitioners and policy makers seeking guidance for clinical decision-making and the formulation of future lung cancer screening strategies for people who have never smoked.</p><p><strong>Systematic review registration: </strong>This protocol has been registered in PROSPERO under the registration number CRD42023483824.</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"8 1","pages":"3"},"PeriodicalIF":0.0,"publicationDate":"2024-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10863273/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139725156","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-06DOI: 10.1186/s41512-024-00165-5
Laura C Rosella, Mackenzie Hurst, Meghan O'Neill, Lief Pagalan, Lori Diemert, Kathy Kornas, Andy Hong, Stacey Fisher, Douglas G Manuel
Introduction: Avoidable hospitalizations are considered preventable given effective and timely primary care management and are an important indicator of health system performance. The ability to predict avoidable hospitalizations at the population level represents a significant advantage for health system decision-makers that could facilitate proactive intervention for ambulatory care-sensitive conditions (ACSCs). The aim of this study is to develop and validate the Avoidable Hospitalization Population Risk Tool (AvHPoRT) that will predict the 5-year risk of first avoidable hospitalization for seven ACSCs using self-reported, routinely collected population health survey data.
Methods and analysis: The derivation cohort will consist of respondents to the first 3 cycles (2000/01, 2003/04, 2005/06) of the Canadian Community Health Survey (CCHS) who are 18-74 years of age at survey administration and a hold-out data set will be used for external validation. Outcome information on avoidable hospitalizations for 5 years following the CCHS interview will be assessed through data linkage to the Discharge Abstract Database (1999/2000-2017/2018) for an estimated sample size of 394,600. Candidate predictor variables will include demographic characteristics, socioeconomic status, self-perceived health measures, health behaviors, chronic conditions, and area-based measures. Sex-specific algorithms will be developed using Weibull accelerated failure time survival models. The model will be validated both using split set cross-validation and external temporal validation split using cycles 2000-2006 compared to 2007-2012. We will assess measures of overall predictive performance (Nagelkerke R2), calibration (calibration plots), and discrimination (Harrell's concordance statistic). Development of the model will be informed by the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) statement.
Ethics and dissemination: This study was approved by the University of Toronto Research Ethics Board. The predictive algorithm and findings from this work will be disseminated at scientific meetings and in peer-reviewed publications.
{"title":"A study protocol for a predictive model to assess population-based avoidable hospitalization risk: Avoidable Hospitalization Population Risk Prediction Tool (AvHPoRT).","authors":"Laura C Rosella, Mackenzie Hurst, Meghan O'Neill, Lief Pagalan, Lori Diemert, Kathy Kornas, Andy Hong, Stacey Fisher, Douglas G Manuel","doi":"10.1186/s41512-024-00165-5","DOIUrl":"10.1186/s41512-024-00165-5","url":null,"abstract":"<p><strong>Introduction: </strong>Avoidable hospitalizations are considered preventable given effective and timely primary care management and are an important indicator of health system performance. The ability to predict avoidable hospitalizations at the population level represents a significant advantage for health system decision-makers that could facilitate proactive intervention for ambulatory care-sensitive conditions (ACSCs). The aim of this study is to develop and validate the Avoidable Hospitalization Population Risk Tool (AvHPoRT) that will predict the 5-year risk of first avoidable hospitalization for seven ACSCs using self-reported, routinely collected population health survey data.</p><p><strong>Methods and analysis: </strong>The derivation cohort will consist of respondents to the first 3 cycles (2000/01, 2003/04, 2005/06) of the Canadian Community Health Survey (CCHS) who are 18-74 years of age at survey administration and a hold-out data set will be used for external validation. Outcome information on avoidable hospitalizations for 5 years following the CCHS interview will be assessed through data linkage to the Discharge Abstract Database (1999/2000-2017/2018) for an estimated sample size of 394,600. Candidate predictor variables will include demographic characteristics, socioeconomic status, self-perceived health measures, health behaviors, chronic conditions, and area-based measures. Sex-specific algorithms will be developed using Weibull accelerated failure time survival models. The model will be validated both using split set cross-validation and external temporal validation split using cycles 2000-2006 compared to 2007-2012. We will assess measures of overall predictive performance (Nagelkerke R<sup>2</sup>), calibration (calibration plots), and discrimination (Harrell's concordance statistic). Development of the model will be informed by the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) statement.</p><p><strong>Ethics and dissemination: </strong>This study was approved by the University of Toronto Research Ethics Board. The predictive algorithm and findings from this work will be disseminated at scientific meetings and in peer-reviewed publications.</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"8 1","pages":"2"},"PeriodicalIF":0.0,"publicationDate":"2024-02-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10845544/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139693616","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-01-23DOI: 10.1186/s41512-024-00164-6
Lana Biot, Laura Jacxsens, Emilie Cardon, Huib Versnel, Koenraad S Rhebergen, Ralf A Boerboom, Annick Gilles, Vincent Van Rompaey, Marc J W Lammers
Background: Speech perception tests are essential to measure the functional use of hearing and to determine the effectiveness of hearing aids and implantable auditory devices. However, these language-based tests require active participation and are influenced by linguistic and neurocognitive skills limiting their use in patients with insufficient language proficiency, cognitive impairment, or in children. We recently developed a non-attentive and objective speech perception prediction model: the Acoustic Change Complex (ACC) prediction model. The ACC prediction model uses electroencephalography to measure alterations in cortical auditory activity caused by frequency changes. The aim is to validate this model in a large-scale external validation study in adult patients with varying degrees of sensorineural hearing loss (SNHL) to confirm the high predictive value of the ACC model and to assess its test-retest reliability.
Methods: A total of 80 participants, aged 18-65 years, will be enrolled in the study. The categories of severity of hearing loss will be used as a blocking factor to establish an equal distribution of patients with various degrees of sensorineural hearing loss. During the first visit, pure tone audiometry, speech in noise tests, a phoneme discrimination test, and the first ACC measurement will be performed. During the second visit (after 1-4 weeks), the same ACC measurement will be performed to assess the test-retest reliability. The acoustic change stimuli for ACC measurements consist of a reference tone with a base frequency of 1000, 2000, or 4000 Hz with a duration of 3000 ms, gliding to a 300-ms target tone with a frequency that is 12% higher than the base frequency. The primary outcome measures are (1) the level of agreement between the predicted speech reception threshold (SRT) and the behavioral SRT, and (2) the level of agreement between the SRT calculated by the first ACC measurement and the SRT of the second ACC measurement. Level of agreement will be assessed with Bland-Altman plots.
Discussion: Previous studies by our group have shown the high predictive value of the ACC model. The successful validation of this model as an effective and reliable biomarker of speech perception will directly benefit the general population, as it will increase the accuracy of hearing evaluations and improve access to adequate hearing rehabilitation.
{"title":"Validation of the acoustic change complex (ACC) prediction model to predict speech perception in noise in adult patients with hearing loss: a study protocol.","authors":"Lana Biot, Laura Jacxsens, Emilie Cardon, Huib Versnel, Koenraad S Rhebergen, Ralf A Boerboom, Annick Gilles, Vincent Van Rompaey, Marc J W Lammers","doi":"10.1186/s41512-024-00164-6","DOIUrl":"10.1186/s41512-024-00164-6","url":null,"abstract":"<p><strong>Background: </strong>Speech perception tests are essential to measure the functional use of hearing and to determine the effectiveness of hearing aids and implantable auditory devices. However, these language-based tests require active participation and are influenced by linguistic and neurocognitive skills limiting their use in patients with insufficient language proficiency, cognitive impairment, or in children. We recently developed a non-attentive and objective speech perception prediction model: the Acoustic Change Complex (ACC) prediction model. The ACC prediction model uses electroencephalography to measure alterations in cortical auditory activity caused by frequency changes. The aim is to validate this model in a large-scale external validation study in adult patients with varying degrees of sensorineural hearing loss (SNHL) to confirm the high predictive value of the ACC model and to assess its test-retest reliability.</p><p><strong>Methods: </strong>A total of 80 participants, aged 18-65 years, will be enrolled in the study. The categories of severity of hearing loss will be used as a blocking factor to establish an equal distribution of patients with various degrees of sensorineural hearing loss. During the first visit, pure tone audiometry, speech in noise tests, a phoneme discrimination test, and the first ACC measurement will be performed. During the second visit (after 1-4 weeks), the same ACC measurement will be performed to assess the test-retest reliability. The acoustic change stimuli for ACC measurements consist of a reference tone with a base frequency of 1000, 2000, or 4000 Hz with a duration of 3000 ms, gliding to a 300-ms target tone with a frequency that is 12% higher than the base frequency. The primary outcome measures are (1) the level of agreement between the predicted speech reception threshold (SRT) and the behavioral SRT, and (2) the level of agreement between the SRT calculated by the first ACC measurement and the SRT of the second ACC measurement. Level of agreement will be assessed with Bland-Altman plots.</p><p><strong>Discussion: </strong>Previous studies by our group have shown the high predictive value of the ACC model. The successful validation of this model as an effective and reliable biomarker of speech perception will directly benefit the general population, as it will increase the accuracy of hearing evaluations and improve access to adequate hearing rehabilitation.</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"8 1","pages":"1"},"PeriodicalIF":0.0,"publicationDate":"2024-01-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10807087/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139543740","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-07DOI: 10.1186/s41512-023-00161-1
Merijn H Rijk, Tamara N Platteel, Geert-Jan Geersing, Monika Hollander, Bert L G P Dalmolen, Paul Little, Frans H Rutten, Maarten van Smeden, Roderick P Venekamp
Background: Community-acquired lower respiratory tract infections (LRTI) are common in primary care and patients at particular risk of adverse outcomes, e.g., hospitalisation and mortality, are challenging to identify. LRTIs are also linked to an increased incidence of cardiovascular diseases (CVD) following the initial infection, whereas concurrent CVD might negatively impact overall prognosis in LRTI patients. Accurate risk prediction of adverse outcomes in LRTI patients, while considering the interplay with CVD, can aid general practitioners (GP) in the clinical decision-making process, and may allow for early detection of deterioration. This paper therefore presents the design of the development and external validation of two models for predicting individual risk of all-cause hospitalisation or mortality (model 1) and short-term incidence of CVD (model 2) in adults presenting to primary care with LRTI.
Methods: Both models will be developed using linked routine electronic health records (EHR) data from Dutch primary and secondary care, and the mortality registry. Adults aged ≥ 40 years with a GP-diagnosis of LRTI between 2016 and 2019 are eligible for inclusion. Relevant patient demographics, medical history, medication use, presenting signs and symptoms, and vital and laboratory measurements will be considered as candidate predictors. Outcomes of interest include 30-day all-cause hospitalisation or mortality (model 1) and 90-day CVD (model 2). Multivariable elastic net regression techniques will be used for model development. During the modelling process, the incremental predictive value of CVD for hospitalisation or all-cause mortality (model 1) will also be assessed. The models will be validated through internal-external cross-validation and external validation in an equivalent cohort of primary care LRTI patients.
Discussion: Implementation of currently available prediction models for primary care LRTI patients is hampered by limited assessment of model performance. While considering the role of CVD in LRTI prognosis, we aim to develop and externally validate two models that predict clinically relevant outcomes to aid GPs in clinical decision-making. Challenges that we anticipate include the possibility of low event rates and common problems related to the use of EHR data, such as candidate predictor measurement and missingness, how best to retrieve information from free text fields, and potential misclassification of outcome events.
{"title":"Predicting adverse outcomes in adults with a community-acquired lower respiratory tract infection: a protocol for the development and validation of two prediction models for (i) all-cause hospitalisation and mortality and (ii) cardiovascular outcomes.","authors":"Merijn H Rijk, Tamara N Platteel, Geert-Jan Geersing, Monika Hollander, Bert L G P Dalmolen, Paul Little, Frans H Rutten, Maarten van Smeden, Roderick P Venekamp","doi":"10.1186/s41512-023-00161-1","DOIUrl":"10.1186/s41512-023-00161-1","url":null,"abstract":"<p><strong>Background: </strong>Community-acquired lower respiratory tract infections (LRTI) are common in primary care and patients at particular risk of adverse outcomes, e.g., hospitalisation and mortality, are challenging to identify. LRTIs are also linked to an increased incidence of cardiovascular diseases (CVD) following the initial infection, whereas concurrent CVD might negatively impact overall prognosis in LRTI patients. Accurate risk prediction of adverse outcomes in LRTI patients, while considering the interplay with CVD, can aid general practitioners (GP) in the clinical decision-making process, and may allow for early detection of deterioration. This paper therefore presents the design of the development and external validation of two models for predicting individual risk of all-cause hospitalisation or mortality (model 1) and short-term incidence of CVD (model 2) in adults presenting to primary care with LRTI.</p><p><strong>Methods: </strong>Both models will be developed using linked routine electronic health records (EHR) data from Dutch primary and secondary care, and the mortality registry. Adults aged ≥ 40 years with a GP-diagnosis of LRTI between 2016 and 2019 are eligible for inclusion. Relevant patient demographics, medical history, medication use, presenting signs and symptoms, and vital and laboratory measurements will be considered as candidate predictors. Outcomes of interest include 30-day all-cause hospitalisation or mortality (model 1) and 90-day CVD (model 2). Multivariable elastic net regression techniques will be used for model development. During the modelling process, the incremental predictive value of CVD for hospitalisation or all-cause mortality (model 1) will also be assessed. The models will be validated through internal-external cross-validation and external validation in an equivalent cohort of primary care LRTI patients.</p><p><strong>Discussion: </strong>Implementation of currently available prediction models for primary care LRTI patients is hampered by limited assessment of model performance. While considering the role of CVD in LRTI prognosis, we aim to develop and externally validate two models that predict clinically relevant outcomes to aid GPs in clinical decision-making. Challenges that we anticipate include the possibility of low event rates and common problems related to the use of EHR data, such as candidate predictor measurement and missingness, how best to retrieve information from free text fields, and potential misclassification of outcome events.</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"7 1","pages":"23"},"PeriodicalIF":0.0,"publicationDate":"2023-12-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10702048/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138500366","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-05DOI: 10.1186/s41512-023-00160-2
David Nickson, Henrik Singmann, Caroline Meyer, Carla Toro, Lukasz Walasek
Background: Recent advances in machine learning combined with the growing availability of digitized health records offer new opportunities for improving early diagnosis of depression. An emerging body of research shows that Electronic Health Records can be used to accurately predict cases of depression on the basis of individual's primary care records. The successes of these studies are undeniable, but there is a growing concern that their results may not be replicable, which could cast doubt on their clinical usefulness.
Methods: To address this issue in the present paper, we set out to reproduce and replicate the work by Nichols et al. (2018), who trained predictive models of depression among young adults using Electronic Healthcare Records. Our contribution consists of three parts. First, we attempt to replicate the methodology used by the original authors, acquiring a more up-to-date set of primary health care records to the same specification and reproducing their data processing and analysis. Second, we test models presented in the original paper on our own data, thus providing out-of-sample prediction of the predictive models. Third, we extend past work by considering several novel machine-learning approaches in an attempt to improve the predictive accuracy achieved in the original work.
Results: In summary, our results demonstrate that the work of Nichols et al. is largely reproducible and replicable. This was the case both for the replication of the original model and the out-of-sample replication applying NRCBM coefficients to our new EHRs data. Although alternative predictive models did not improve model performance over standard logistic regression, our results indicate that stepwise variable selection is not stable even in the case of large data sets.
Conclusion: We discuss the challenges associated with the research on mental health and Electronic Health Records, including the need to produce interpretable and robust models. We demonstrated some potential issues associated with the reliance on EHRs, including changes in the regulations and guidelines (such as the QOF guidelines in the UK) and reliance on visits to GP as a predictor of specific disorders.
{"title":"Replicability and reproducibility of predictive models for diagnosis of depression among young adults using Electronic Health Records.","authors":"David Nickson, Henrik Singmann, Caroline Meyer, Carla Toro, Lukasz Walasek","doi":"10.1186/s41512-023-00160-2","DOIUrl":"10.1186/s41512-023-00160-2","url":null,"abstract":"<p><strong>Background: </strong>Recent advances in machine learning combined with the growing availability of digitized health records offer new opportunities for improving early diagnosis of depression. An emerging body of research shows that Electronic Health Records can be used to accurately predict cases of depression on the basis of individual's primary care records. The successes of these studies are undeniable, but there is a growing concern that their results may not be replicable, which could cast doubt on their clinical usefulness.</p><p><strong>Methods: </strong>To address this issue in the present paper, we set out to reproduce and replicate the work by Nichols et al. (2018), who trained predictive models of depression among young adults using Electronic Healthcare Records. Our contribution consists of three parts. First, we attempt to replicate the methodology used by the original authors, acquiring a more up-to-date set of primary health care records to the same specification and reproducing their data processing and analysis. Second, we test models presented in the original paper on our own data, thus providing out-of-sample prediction of the predictive models. Third, we extend past work by considering several novel machine-learning approaches in an attempt to improve the predictive accuracy achieved in the original work.</p><p><strong>Results: </strong>In summary, our results demonstrate that the work of Nichols et al. is largely reproducible and replicable. This was the case both for the replication of the original model and the out-of-sample replication applying NRCBM coefficients to our new EHRs data. Although alternative predictive models did not improve model performance over standard logistic regression, our results indicate that stepwise variable selection is not stable even in the case of large data sets.</p><p><strong>Conclusion: </strong>We discuss the challenges associated with the research on mental health and Electronic Health Records, including the need to produce interpretable and robust models. We demonstrated some potential issues associated with the reliance on EHRs, including changes in the regulations and guidelines (such as the QOF guidelines in the UK) and reliance on visits to GP as a predictor of specific disorders.</p>","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"7 1","pages":"25"},"PeriodicalIF":0.0,"publicationDate":"2023-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10696659/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138483516","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-01DOI: 10.1186/s41512-023-00163-z
K. Tanner, Ruth H. Keogh, Carol A. C. Coupland, Julia Hippisley-Cox, Karla Diaz-Ordaz
{"title":"Dynamic updating of clinical survival prediction models in a changing environment","authors":"K. Tanner, Ruth H. Keogh, Carol A. C. Coupland, Julia Hippisley-Cox, Karla Diaz-Ordaz","doi":"10.1186/s41512-023-00163-z","DOIUrl":"https://doi.org/10.1186/s41512-023-00163-z","url":null,"abstract":"","PeriodicalId":72800,"journal":{"name":"Diagnostic and prognostic research","volume":"82 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138621840","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}