Pub Date : 2023-12-13eCollection Date: 2023-12-01DOI: 10.1093/jamiaopen/ooad106
Han Yu, Allan F Simpao, Victor M Ruiz, Olivia Nelson, Wallis T Muhly, Tori N Sutherland, Julia A Gálvez, Mykhailo B Pushkar, Paul A Stricker, Fuchiang Rich Tsui
Objectives: Pediatric emergence delirium is an undesirable outcome that is understudied. Development of a predictive model is an initial step toward reducing its occurrence. This study aimed to apply machine learning (ML) methods to a large clinical dataset to develop a predictive model for pediatric emergence delirium.
Materials and methods: We performed a single-center retrospective cohort study using electronic health record data from February 2015 to December 2019. We built and evaluated 4 commonly used ML models for predicting emergence delirium: least absolute shrinkage and selection operator, ridge regression, random forest, and extreme gradient boosting. The primary outcome was the occurrence of emergence delirium, defined as a Watcha score of 3 or 4 recorded at any time during recovery.
Results: The dataset included 54 776 encounters across 43 830 patients. The 4 ML models performed similarly with performance assessed by the area under the receiver operating characteristic curves ranging from 0.74 to 0.75. Notable variables associated with increased risk included adenoidectomy with or without tonsillectomy, decreasing age, midazolam premedication, and ondansetron administration, while intravenous induction and ketorolac were associated with reduced risk of emergence delirium.
Conclusions: Four different ML models demonstrated similar performance in predicting postoperative emergence delirium using a large pediatric dataset. The prediction performance of the models draws attention to our incomplete understanding of this phenomenon based on the studied variables. The results from our modeling could serve as a first step in designing a predictive clinical decision support system, but further optimization and validation are needed.
Clinical trial number and registry url: Not applicable.
{"title":"Predicting pediatric emergence delirium using data-driven machine learning applied to electronic health record dataset at a quaternary care pediatric hospital.","authors":"Han Yu, Allan F Simpao, Victor M Ruiz, Olivia Nelson, Wallis T Muhly, Tori N Sutherland, Julia A Gálvez, Mykhailo B Pushkar, Paul A Stricker, Fuchiang Rich Tsui","doi":"10.1093/jamiaopen/ooad106","DOIUrl":"https://doi.org/10.1093/jamiaopen/ooad106","url":null,"abstract":"<p><strong>Objectives: </strong>Pediatric emergence delirium is an undesirable outcome that is understudied. Development of a predictive model is an initial step toward reducing its occurrence. This study aimed to apply machine learning (ML) methods to a large clinical dataset to develop a predictive model for pediatric emergence delirium.</p><p><strong>Materials and methods: </strong>We performed a single-center retrospective cohort study using electronic health record data from February 2015 to December 2019. We built and evaluated 4 commonly used ML models for predicting emergence delirium: least absolute shrinkage and selection operator, ridge regression, random forest, and extreme gradient boosting. The primary outcome was the occurrence of emergence delirium, defined as a Watcha score of 3 or 4 recorded at any time during recovery.</p><p><strong>Results: </strong>The dataset included 54 776 encounters across 43 830 patients. The 4 ML models performed similarly with performance assessed by the area under the receiver operating characteristic curves ranging from 0.74 to 0.75. Notable variables associated with increased risk included adenoidectomy with or without tonsillectomy, decreasing age, midazolam premedication, and ondansetron administration, while intravenous induction and ketorolac were associated with reduced risk of emergence delirium.</p><p><strong>Conclusions: </strong>Four different ML models demonstrated similar performance in predicting postoperative emergence delirium using a large pediatric dataset. The prediction performance of the models draws attention to our incomplete understanding of this phenomenon based on the studied variables. The results from our modeling could serve as a first step in designing a predictive clinical decision support system, but further optimization and validation are needed.</p><p><strong>Clinical trial number and registry url: </strong>Not applicable.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 4","pages":"ooad106"},"PeriodicalIF":2.1,"publicationDate":"2023-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10719078/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138809941","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-13eCollection Date: 2023-12-01DOI: 10.1093/jamiaopen/ooad104
Priyank Raj, Youmin Cho, Yun Jiang, Yang Gong
Objective: This article provides insight into our process and considerations for selecting patient-reported outcome measures (PROMs) designed for self-reporting symptoms and quality-of-life among breast cancer (BCA) patients undergoing oral anticancer agent treatment via a patient-facing technology (PFT) platform.
Methods: Following established guidelines, we conducted a thorough assessment of a specific set of PROMs, comparing their content to identify the most suitable options for studying BCA patients.
Results: We recommend utilizing the combination of EORTC QLQ-C30 + EORTC QLQ-BR45 as the preferred instrument, especially when developing a dedicated "breast cancer-only" application.
Discussion: When developing and maintaining a dashboard for a PFT platform that includes multiple cancer types, it is important to consider the feasibility of interface design and workload. To achieve this, we recommend using PRO-CTCAE+PROMIS 10 GH for the PFT. Moreover, it is important to consider adding ad hoc items to complement the chosen PROM(s).
Conclusion: This article describes our efforts to identify PROMs for self-reported data while considering patient and developer burdens, providing guidance to PFT developers facing similar challenges in PROM selection.
{"title":"Selecting patient-reported outcome measures for a patient-facing technology.","authors":"Priyank Raj, Youmin Cho, Yun Jiang, Yang Gong","doi":"10.1093/jamiaopen/ooad104","DOIUrl":"10.1093/jamiaopen/ooad104","url":null,"abstract":"<p><strong>Objective: </strong>This article provides insight into our process and considerations for selecting patient-reported outcome measures (PROMs) designed for self-reporting symptoms and quality-of-life among breast cancer (BCA) patients undergoing oral anticancer agent treatment via a patient-facing technology (PFT) platform.</p><p><strong>Methods: </strong>Following established guidelines, we conducted a thorough assessment of a specific set of PROMs, comparing their content to identify the most suitable options for studying BCA patients.</p><p><strong>Results: </strong>We recommend utilizing the combination of EORTC QLQ-C30 + EORTC QLQ-BR45 as the preferred instrument, especially when developing a dedicated \"breast cancer-only\" application.</p><p><strong>Discussion: </strong>When developing and maintaining a dashboard for a PFT platform that includes multiple cancer types, it is important to consider the feasibility of interface design and workload. To achieve this, we recommend using PRO-CTCAE+PROMIS 10 GH for the PFT. Moreover, it is important to consider adding ad hoc items to complement the chosen PROM(s).</p><p><strong>Conclusion: </strong>This article describes our efforts to identify PROMs for self-reported data while considering patient and developer burdens, providing guidance to PFT developers facing similar challenges in PROM selection.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 4","pages":"ooad104"},"PeriodicalIF":2.1,"publicationDate":"2023-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10719077/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138809943","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-05eCollection Date: 2023-12-01DOI: 10.1093/jamiaopen/ooad101
Andrew J Webb, Bayleigh Carver, Sandra Rowe, Andrea Sikora
Objectives: A lack of pharmacist-specific risk-stratification scores in the electronic health record (EHR) may limit resource optimization. The medication regimen complexity-intensive care unit (MRC-ICU) score was implemented into our center's EHR for use by clinical pharmacists. The purpose of this evaluation was to evaluate MRC-ICU as a predictor of pharmacist workload and to assess its potential as an additional dimension to traditional workload measures.
Materials and methods: Data were abstracted from the EHR on adult ICU patients, including MRC-ICU scores and 2 traditional measures of pharmacist workload: numbers of medication orders verified and interventions logged. This was a single-center study of an EHR-integrated MRC-ICU tool. The primary outcome was the association of MRC-ICU with institutional metrics of pharmacist workload. Associations were assessed using the initial 24-h maximum MRC-ICU score's Pearson's correlation with overall admission workload and the day-to-day association using generalized linear mixed-effects modeling.
Results: A total of 1205 patients over 5083 patient-days were evaluated. Baseline MRC-ICU was correlated with both cumulative order volume (Spearman's rho 0.41, P < .001) and cumulative interventions placed (Spearman's rho 0.27, P < .001). A 1-point increase in maximum daily MRC-ICU was associated with a 31% increase in order volume (95% CI, 24%-38%) and 4% increase in interventions (95% CI, 2%-5%).
Discussion and conclusion: The MRC-ICU is a validated score that has been previously correlated with important patient-centered outcomes. Here, MRC-ICU was modestly associated with 2 traditional objective measures of pharmacist workload, including orders verified and interventions placed, which is an important step for its use as a tool for resource utilization needs.
{"title":"The use of electronic health record embedded MRC-ICU as a metric for critical care pharmacist workload.","authors":"Andrew J Webb, Bayleigh Carver, Sandra Rowe, Andrea Sikora","doi":"10.1093/jamiaopen/ooad101","DOIUrl":"10.1093/jamiaopen/ooad101","url":null,"abstract":"<p><strong>Objectives: </strong>A lack of pharmacist-specific risk-stratification scores in the electronic health record (EHR) may limit resource optimization. The medication regimen complexity-intensive care unit (MRC-ICU) score was implemented into our center's EHR for use by clinical pharmacists. The purpose of this evaluation was to evaluate MRC-ICU as a predictor of pharmacist workload and to assess its potential as an additional dimension to traditional workload measures.</p><p><strong>Materials and methods: </strong>Data were abstracted from the EHR on adult ICU patients, including MRC-ICU scores and 2 traditional measures of pharmacist workload: numbers of medication orders verified and interventions logged. This was a single-center study of an EHR-integrated MRC-ICU tool. The primary outcome was the association of MRC-ICU with institutional metrics of pharmacist workload. Associations were assessed using the initial 24-h maximum MRC-ICU score's Pearson's correlation with overall admission workload and the day-to-day association using generalized linear mixed-effects modeling.</p><p><strong>Results: </strong>A total of 1205 patients over 5083 patient-days were evaluated. Baseline MRC-ICU was correlated with both cumulative order volume (Spearman's rho 0.41, <i>P</i> < .001) and cumulative interventions placed (Spearman's rho 0.27, <i>P</i> < .001). A 1-point increase in maximum daily MRC-ICU was associated with a 31% increase in order volume (95% CI, 24%-38%) and 4% increase in interventions (95% CI, 2%-5%).</p><p><strong>Discussion and conclusion: </strong>The MRC-ICU is a validated score that has been previously correlated with important patient-centered outcomes. Here, MRC-ICU was modestly associated with 2 traditional objective measures of pharmacist workload, including orders verified and interventions placed, which is an important step for its use as a tool for resource utilization needs.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 4","pages":"ooad101"},"PeriodicalIF":2.1,"publicationDate":"2023-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10697785/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138499661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-05eCollection Date: 2023-12-01DOI: 10.1093/jamiaopen/ooad100
Marek Oja, Sirli Tamm, Kerli Mooses, Maarja Pajusalu, Harry-Anton Talvik, Anne Ott, Marianna Laht, Maria Malk, Marcus Lõo, Johannes Holm, Markus Haug, Hendrik Šuvalov, Dage Särg, Jaak Vilo, Sven Laur, Raivo Kolde, Sulev Reisberg
Objective: To describe the reusable transformation process of electronic health records (EHR), claims, and prescriptions data into Observational Medical Outcome Partnership (OMOP) Common Data Model (CDM), together with challenges faced and solutions implemented.
Materials and methods: We used Estonian national health databases that store almost all residents' claims, prescriptions, and EHR records. To develop and demonstrate the transformation process of Estonian health data to OMOP CDM, we used a 10% random sample of the Estonian population (n = 150 824 patients) from 2012 to 2019 (MAITT dataset). For the sample, complete information from all 3 databases was converted to OMOP CDM version 5.3. The validation was performed using open-source tools.
Results: In total, we transformed over 100 million entries to standard concepts using standard OMOP vocabularies with the average mapping rate 95%. For conditions, observations, drugs, and measurements, the mapping rate was over 90%. In most cases, SNOMED Clinical Terms were used as the target vocabulary.
Discussion: During the transformation process, we encountered several challenges, which are described in detail with concrete examples and solutions.
Conclusion: For a representative 10% random sample, we successfully transferred complete records from 3 national health databases to OMOP CDM and created a reusable transformation process. Our work helps future researchers to transform linked databases into OMOP CDM more efficiently, ultimately leading to better real-world evidence.
{"title":"Transforming Estonian health data to the Observational Medical Outcomes Partnership (OMOP) Common Data Model: lessons learned.","authors":"Marek Oja, Sirli Tamm, Kerli Mooses, Maarja Pajusalu, Harry-Anton Talvik, Anne Ott, Marianna Laht, Maria Malk, Marcus Lõo, Johannes Holm, Markus Haug, Hendrik Šuvalov, Dage Särg, Jaak Vilo, Sven Laur, Raivo Kolde, Sulev Reisberg","doi":"10.1093/jamiaopen/ooad100","DOIUrl":"10.1093/jamiaopen/ooad100","url":null,"abstract":"<p><strong>Objective: </strong>To describe the reusable transformation process of electronic health records (EHR), claims, and prescriptions data into Observational Medical Outcome Partnership (OMOP) Common Data Model (CDM), together with challenges faced and solutions implemented.</p><p><strong>Materials and methods: </strong>We used Estonian national health databases that store almost all residents' claims, prescriptions, and EHR records. To develop and demonstrate the transformation process of Estonian health data to OMOP CDM, we used a 10% random sample of the Estonian population (<i>n</i> = 150 824 patients) from 2012 to 2019 (MAITT dataset). For the sample, complete information from all 3 databases was converted to OMOP CDM version 5.3. The validation was performed using open-source tools.</p><p><strong>Results: </strong>In total, we transformed over 100 million entries to standard concepts using standard OMOP vocabularies with the average mapping rate 95%. For conditions, observations, drugs, and measurements, the mapping rate was over 90%. In most cases, SNOMED Clinical Terms were used as the target vocabulary.</p><p><strong>Discussion: </strong>During the transformation process, we encountered several challenges, which are described in detail with concrete examples and solutions.</p><p><strong>Conclusion: </strong>For a representative 10% random sample, we successfully transferred complete records from 3 national health databases to OMOP CDM and created a reusable transformation process. Our work helps future researchers to transform linked databases into OMOP CDM more efficiently, ultimately leading to better real-world evidence.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 4","pages":"ooad100"},"PeriodicalIF":2.1,"publicationDate":"2023-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10697784/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138499662","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-01DOI: 10.1093/jamiaopen/ooad105
Rashaud Senior, Lisa Pickett, Andrew Stirling, Shwetha Dash, Patti Gorgone, Georgina Durst, Debra Jones, Richard Shannon, N. Bhavsar, Armando Bedoya
Abstract Introduction Gun violence remains a concerning and persistent issue in our country. Novel dashboards may integrate and summarize important clinical and non-clinical data that can inform targeted interventions to address the underlying causes of gun violence. Methods Data from various clinical and non-clinical sources were sourced, cleaned, and integrated into a customizable dashboard that summarizes and provides insight into the underlying factors that impact local gun violence episodes. Results The dashboards contained data from 7786 encounters and 1152 distinct patients from our Emergency Department’s Trauma Registry with various patterns noted by the team. A multidisciplinary executive team, including subject matter experts in community-based interventions, epidemiology, and social sciences, was formed to design targeted interventions based on these observations. Conclusion Targeted interventions to reduce gun violence require a multimodal data sourcing and standardization approach, the inclusion of neighborhood-level data, and a dedicated multidisciplinary team to act on the generated insights.
{"title":"Development of an interactive dashboard for gun violence pattern analysis and intervention design at the local level","authors":"Rashaud Senior, Lisa Pickett, Andrew Stirling, Shwetha Dash, Patti Gorgone, Georgina Durst, Debra Jones, Richard Shannon, N. Bhavsar, Armando Bedoya","doi":"10.1093/jamiaopen/ooad105","DOIUrl":"https://doi.org/10.1093/jamiaopen/ooad105","url":null,"abstract":"Abstract Introduction Gun violence remains a concerning and persistent issue in our country. Novel dashboards may integrate and summarize important clinical and non-clinical data that can inform targeted interventions to address the underlying causes of gun violence. Methods Data from various clinical and non-clinical sources were sourced, cleaned, and integrated into a customizable dashboard that summarizes and provides insight into the underlying factors that impact local gun violence episodes. Results The dashboards contained data from 7786 encounters and 1152 distinct patients from our Emergency Department’s Trauma Registry with various patterns noted by the team. A multidisciplinary executive team, including subject matter experts in community-based interventions, epidemiology, and social sciences, was formed to design targeted interventions based on these observations. Conclusion Targeted interventions to reduce gun violence require a multimodal data sourcing and standardization approach, the inclusion of neighborhood-level data, and a dedicated multidisciplinary team to act on the generated insights.","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":" 16","pages":""},"PeriodicalIF":2.1,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138611536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-01DOI: 10.1093/jamiaopen/ooad109
G. Karway, J. Koyner, John Caskey, Alexandra B Spicer, Kyle A. Carey, Emily R. Gilbert, D. Dligach, A. Mayampurath, Majid Afshar, M. Churpek
Abstract Objectives To develop and externally validate machine learning models using structured and unstructured electronic health record data to predict postoperative acute kidney injury (AKI) across inpatient settings. Materials and Methods Data for adult postoperative admissions to the Loyola University Medical Center (2009-2017) were used for model development and admissions to the University of Wisconsin-Madison (2009-2020) were used for validation. Structured features included demographics, vital signs, laboratory results, and nurse-documented scores. Unstructured text from clinical notes were converted into concept unique identifiers (CUIs) using the clinical Text Analysis and Knowledge Extraction System. The primary outcome was the development of Kidney Disease Improvement Global Outcomes stage 2 AKI within 7 days after leaving the operating room. We derived unimodal extreme gradient boosting machines (XGBoost) and elastic net logistic regression (GLMNET) models using structured-only data and multimodal models combining structured data with CUI features. Model comparison was performed using the receiver operating characteristic curve (AUROC), with Delong’s test for statistical differences. Results The study cohort included 138 389 adult patient admissions (mean [SD] age 58 [16] years; 11 506 [8%] African-American; and 70 826 [51%] female) across the 2 sites. Of those, 2959 (2.1%) developed stage 2 AKI or higher. Across all data types, XGBoost outperformed GLMNET (mean AUROC 0.81 [95% confidence interval (CI), 0.80-0.82] vs 0.78 [95% CI, 0.77-0.79]). The multimodal XGBoost model incorporating CUIs parameterized as term frequency-inverse document frequency (TF-IDF) showed the highest discrimination performance (AUROC 0.82 [95% CI, 0.81-0.83]) over unimodal models (AUROC 0.79 [95% CI, 0.78-0.80]). Discussion A multimodality approach with structured data and TF-IDF weighting of CUIs increased model performance over structured data-only models. Conclusion These findings highlight the predictive power of CUIs when merged with structured data for clinical prediction models, which may improve the detection of postoperative AKI.
摘要 目的 利用结构化和非结构化电子健康记录数据开发和外部验证机器学习模型,以预测不同住院环境下的术后急性肾损伤(AKI)。材料与方法 洛约拉大学医学中心(2009-2017 年)的成人术后入院数据用于模型开发,威斯康星大学麦迪逊分校(2009-2020 年)的入院数据用于验证。结构化特征包括人口统计学、生命体征、实验室结果和护士记录的评分。临床笔记中的非结构化文本通过临床文本分析和知识提取系统转换为概念唯一标识符(CUI)。主要结果是在离开手术室后 7 天内出现肾病改善全球结果 2 期 AKI。我们利用纯结构化数据推导出了单模态极端梯度提升机(XGBoost)和弹性网逻辑回归(GLMNET)模型,并结合结构化数据和 CUI 特征推导出了多模态模型。模型比较采用接收者操作特征曲线 (AUROC),并通过德龙检验法进行统计学差异检验。结果 研究队列包括两个地点收治的 138 389 名成年患者(平均 [SD] 年龄 58 [16] 岁;11 506 [8%] 非洲裔美国人;70 826 [51%] 女性)。其中 2959 人(2.1%)发展为 2 期 AKI 或以上。在所有数据类型中,XGBoost 的表现均优于 GLMNET(平均 AUROC 为 0.81 [95% 置信区间 (CI),0.80-0.82] vs 0.78 [95% CI,0.77-0.79])。与单模态模型(AUROC 0.79 [95% CI, 0.78-0.80])相比,以词频-反文档频率(TF-IDF)为参数的 CUI 多模态 XGBoost 模型显示出最高的识别性能(AUROC 0.82 [95% CI, 0.81-0.83])。讨论 与仅使用结构化数据的模型相比,使用结构化数据和 TF-IDF 加权 CUI 的多模态方法提高了模型性能。结论 这些研究结果凸显了 CUI 与结构化数据合并用于临床预测模型时的预测能力,这可能会改善术后 AKI 的检测。
{"title":"Development and external validation of multimodal postoperative acute kidney injury risk machine learning models","authors":"G. Karway, J. Koyner, John Caskey, Alexandra B Spicer, Kyle A. Carey, Emily R. Gilbert, D. Dligach, A. Mayampurath, Majid Afshar, M. Churpek","doi":"10.1093/jamiaopen/ooad109","DOIUrl":"https://doi.org/10.1093/jamiaopen/ooad109","url":null,"abstract":"Abstract Objectives To develop and externally validate machine learning models using structured and unstructured electronic health record data to predict postoperative acute kidney injury (AKI) across inpatient settings. Materials and Methods Data for adult postoperative admissions to the Loyola University Medical Center (2009-2017) were used for model development and admissions to the University of Wisconsin-Madison (2009-2020) were used for validation. Structured features included demographics, vital signs, laboratory results, and nurse-documented scores. Unstructured text from clinical notes were converted into concept unique identifiers (CUIs) using the clinical Text Analysis and Knowledge Extraction System. The primary outcome was the development of Kidney Disease Improvement Global Outcomes stage 2 AKI within 7 days after leaving the operating room. We derived unimodal extreme gradient boosting machines (XGBoost) and elastic net logistic regression (GLMNET) models using structured-only data and multimodal models combining structured data with CUI features. Model comparison was performed using the receiver operating characteristic curve (AUROC), with Delong’s test for statistical differences. Results The study cohort included 138 389 adult patient admissions (mean [SD] age 58 [16] years; 11 506 [8%] African-American; and 70 826 [51%] female) across the 2 sites. Of those, 2959 (2.1%) developed stage 2 AKI or higher. Across all data types, XGBoost outperformed GLMNET (mean AUROC 0.81 [95% confidence interval (CI), 0.80-0.82] vs 0.78 [95% CI, 0.77-0.79]). The multimodal XGBoost model incorporating CUIs parameterized as term frequency-inverse document frequency (TF-IDF) showed the highest discrimination performance (AUROC 0.82 [95% CI, 0.81-0.83]) over unimodal models (AUROC 0.79 [95% CI, 0.78-0.80]). Discussion A multimodality approach with structured data and TF-IDF weighting of CUIs increased model performance over structured data-only models. Conclusion These findings highlight the predictive power of CUIs when merged with structured data for clinical prediction models, which may improve the detection of postoperative AKI.","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"60 24","pages":""},"PeriodicalIF":2.1,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138985926","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-28eCollection Date: 2023-12-01DOI: 10.1093/jamiaopen/ooad094
Abhishek Dhankar, Alan Katz
Objectives: Present an artificial intelligence-enabled pipeline for estimating the prevalence of depression and general anxiety among pregnant women using texts from their social media posts. Use said pipeline to analyze mental health trends on subreddits frequented by pregnant women and report on interesting insights that could be helpful for policy-makers, clinicians, etc.
Materials and methods: We used pretrained transformer-based models to build a natural language processing pipeline that can automatically detect depressed pregnant women on social media and carry out topic modeling to detect their concerns.
Results: We detected depressed posts by pregnant women on Reddit and validated the performance of the depression classification model by carrying out topic modeling to reveal that depressive topics were detected. The proportion of potentially depressed surprisingly reduced during the pandemic (2020 and 2021). Queries related to antidepressants, such as Zoloft, and potential ways of managing mental health dominated discourse before the pandemic (2018 and 2019), whereas queries about pelvic pain and associated stress dominated the discourse during the pandemic.
Discussion and conclusion: Supportive online communities could be a factor in alleviating stress related to the pandemic, hence the reduction in the proportion of depressed users during the pandemic. Stress during the pandemic has been associated with pelvic pain among pregnant women, and this trend is confirmed through topic modeling of depressive posts during the pandemic.
{"title":"Tracking pregnant women's mental health through social media: an analysis of reddit posts.","authors":"Abhishek Dhankar, Alan Katz","doi":"10.1093/jamiaopen/ooad094","DOIUrl":"10.1093/jamiaopen/ooad094","url":null,"abstract":"<p><strong>Objectives: </strong>Present an artificial intelligence-enabled pipeline for estimating the prevalence of depression and general anxiety among pregnant women using texts from their social media posts. Use said pipeline to analyze mental health trends on subreddits frequented by pregnant women and report on interesting insights that could be helpful for policy-makers, clinicians, etc.</p><p><strong>Materials and methods: </strong>We used pretrained transformer-based models to build a natural language processing pipeline that can automatically detect depressed pregnant women on social media and carry out topic modeling to detect their concerns.</p><p><strong>Results: </strong>We detected depressed posts by pregnant women on Reddit and validated the performance of the depression classification model by carrying out topic modeling to reveal that depressive topics were detected. The proportion of potentially depressed surprisingly reduced during the pandemic (2020 and 2021). Queries related to antidepressants, such as Zoloft, and potential ways of managing mental health dominated discourse before the pandemic (2018 and 2019), whereas queries about pelvic pain and associated stress dominated the discourse during the pandemic.</p><p><strong>Discussion and conclusion: </strong>Supportive online communities could be a factor in alleviating stress related to the pandemic, hence the reduction in the proportion of depressed users during the pandemic. Stress during the pandemic has been associated with pelvic pain among pregnant women, and this trend is confirmed through topic modeling of depressive posts during the pandemic.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 4","pages":"ooad094"},"PeriodicalIF":2.5,"publicationDate":"2023-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10684261/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138463164","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-28eCollection Date: 2023-12-01DOI: 10.1093/jamiaopen/ooad099
Thomas H Payne, Grace K Turner
Objectives: We describe an automated transcription system that addresses many documentation problems and fits within scheduled clinical hours.
Materials and methods: During visits, the provider listens to the patient while maintaining eye contact and making brief notes on paper. Immediately after the visit conclusion and before the next, the provider makes a short voice recording on a smartphone which is transmitted to the system. The system uses a public domain general language model, and a hypertuned provider-specific language model that is iteratively refined as each produced note is edited by the physician, followed by final automated processing steps to add any templated text to the note.
Results: The provider leaves the clinic having completed all voice files, median duration 3.4 minutes. Created notes are formatted as preferred and are a median of 363 words (range 125-1175).
Discussion: This approach permits documentation to occur almost entirely within scheduled clinic hours, without copy-forward errors, and without interference with patient-provider interaction.
Conclusion: Though no documentation method is likely to appeal to all, this approach may appeal to many physicians and avoid many current problems with documentation.
{"title":"I'm not burned out. This is how I write notes.","authors":"Thomas H Payne, Grace K Turner","doi":"10.1093/jamiaopen/ooad099","DOIUrl":"10.1093/jamiaopen/ooad099","url":null,"abstract":"<p><strong>Objectives: </strong>We describe an automated transcription system that addresses many documentation problems and fits within scheduled clinical hours.</p><p><strong>Materials and methods: </strong>During visits, the provider listens to the patient while maintaining eye contact and making brief notes on paper. Immediately after the visit conclusion and before the next, the provider makes a short voice recording on a smartphone which is transmitted to the system. The system uses a public domain general language model, and a hypertuned provider-specific language model that is iteratively refined as each produced note is edited by the physician, followed by final automated processing steps to add any templated text to the note.</p><p><strong>Results: </strong>The provider leaves the clinic having completed all voice files, median duration 3.4 minutes. Created notes are formatted as preferred and are a median of 363 words (range 125-1175).</p><p><strong>Discussion: </strong>This approach permits documentation to occur almost entirely within scheduled clinic hours, without copy-forward errors, and without interference with patient-provider interaction.</p><p><strong>Conclusion: </strong>Though no documentation method is likely to appeal to all, this approach may appeal to many physicians and avoid many current problems with documentation.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 4","pages":"ooad099"},"PeriodicalIF":2.1,"publicationDate":"2023-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10684266/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138463162","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-22eCollection Date: 2023-12-01DOI: 10.1093/jamiaopen/ooad103
Chelsea Richwine, Jordan Everson, Vaishali Patel
Objective: To understand whether hospitals had electronic access to information needed to treat COVID-19 patients and identify factors contributing to differences in information availability.
Materials and methods: Using 2021 data from the American Hospital Association IT Supplement, we produced national estimates on the electronic availability of information needed to treat COVID-19 at US non-federal acute care hospitals (N = 1976) and assessed differences in information availability by hospital characteristics and engagement in interoperable exchange.
Results: In 2021, 38% of hospitals electronically received information needed to effectively treat COVID-19 patients. Information availability was significantly higher among higher-resourced hospitals and those engaged in interoperable exchange (44%) compared to their counterparts. In adjusted analyses, hospitals engaged in interoperable exchange were 140% more likely to receive needed information electronically compared to those not engaged in exchange (relative risk [RR]=2.40, 95% CI, 1.82-3.17, P<.001). System member hospitals (RR = 1.62, 95% CI, 1.36-1.92, P<.001) and major teaching hospitals (RR = 1.35, 95% CI, 1.10-1.64, P=.004) were more likely to have information available; for-profit hospitals (RR = 0.14, 95% CI, 0.08-0.24, P<.001) and hospitals in high social deprivation areas (RR = 0.83, 95% CI, 0.71-0.98, P = .02) were less likely to have information available.
Discussion: Despite high rates of hospitals' engagement in interoperable exchange, hospitals' electronic access to information needed to support the care of COVID-19 patients was limited.
Conclusion: Limited electronic access to patient information from outside sources may impede hospitals' ability to effectively treat COVID-19 and support patient care during public health emergencies.
{"title":"Hospitals' electronic access to information needed to treat COVID-19.","authors":"Chelsea Richwine, Jordan Everson, Vaishali Patel","doi":"10.1093/jamiaopen/ooad103","DOIUrl":"10.1093/jamiaopen/ooad103","url":null,"abstract":"<p><strong>Objective: </strong>To understand whether hospitals had electronic access to information needed to treat COVID-19 patients and identify factors contributing to differences in information availability.</p><p><strong>Materials and methods: </strong>Using 2021 data from the American Hospital Association IT Supplement, we produced national estimates on the electronic availability of information needed to treat COVID-19 at US non-federal acute care hospitals (<i>N</i> = 1976) and assessed differences in information availability by hospital characteristics and engagement in interoperable exchange.</p><p><strong>Results: </strong>In 2021, 38% of hospitals electronically received information needed to effectively treat COVID-19 patients. Information availability was significantly higher among higher-resourced hospitals and those engaged in interoperable exchange (44%) compared to their counterparts. In adjusted analyses, hospitals engaged in interoperable exchange were 140% more likely to receive needed information electronically compared to those not engaged in exchange (relative risk [RR]=2.40, 95% CI, 1.82-3.17, <i>P</i><.001). System member hospitals (RR = 1.62, 95% CI, 1.36-1.92, <i>P</i><.001) and major teaching hospitals (RR = 1.35, 95% CI, 1.10-1.64, <i>P</i>=.004) were more likely to have information available; for-profit hospitals (RR = 0.14, 95% CI, 0.08-0.24, <i>P</i><.001) and hospitals in high social deprivation areas (RR = 0.83, 95% CI, 0.71-0.98, <i>P</i> = .02) were less likely to have information available.</p><p><strong>Discussion: </strong>Despite high rates of hospitals' engagement in interoperable exchange, hospitals' electronic access to information needed to support the care of COVID-19 patients was limited.</p><p><strong>Conclusion: </strong>Limited electronic access to patient information from outside sources may impede hospitals' ability to effectively treat COVID-19 and support patient care during public health emergencies.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 4","pages":"ooad103"},"PeriodicalIF":2.1,"publicationDate":"2023-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10684259/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138463161","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-21eCollection Date: 2023-12-01DOI: 10.1093/jamiaopen/ooad096
Rupa Makadia, Azza Shoaibi, Gowtham A Rao, Anna Ostropolets, Peter R Rijnbeek, Erica A Voss, Talita Duarte-Salles, Juan Manuel Ramírez-Anguita, Miguel A Mayer, Filip Maljković, Spiros Denaxas, Fredrik Nyberg, Vaclav Papez, Anthony G Sena, Thamir M Alshammari, Lana Y H Lai, Kevin Haynes, Marc A Suchard, George Hripcsak, Patrick B Ryan
Objective: Developing accurate phenotype definitions is critical in obtaining reliable and reproducible background rates in safety research. This study aims to illustrate the differences in background incidence rates by comparing definitions for a given outcome.
Materials and methods: We used 16 data sources to systematically generate and evaluate outcomes for 13 adverse events and their overall background rates. We examined the effect of different modifications (inpatient setting, standardization of code set, and code set changes) to the computable phenotype on background incidence rates.
Results: Rate ratios (RRs) of the incidence rates from each computable phenotype definition varied across outcomes, with inpatient restriction showing the highest variation from 1 to 11.93. Standardization of code set RRs ranges from 1 to 1.64, and code set changes range from 1 to 2.52.
Discussion: The modification that has the highest impact is requiring inpatient place of service, leading to at least a 2-fold higher incidence rate in the base definition. Standardization showed almost no change when using source code variations. The strength of the effect in the inpatient restriction is highly dependent on the outcome. Changing definitions from broad to narrow showed the most variability by age/gender/database across phenotypes and less than a 2-fold increase in rate compared to the base definition.
Conclusion: Characterization of outcomes across a network of databases yields insights into sensitivity and specificity trade-offs when definitions are altered. Outcomes should be thoroughly evaluated prior to use for background rates for their plausibility for use across a global network.
{"title":"Evaluating the impact of alternative phenotype definitions on incidence rates across a global data network.","authors":"Rupa Makadia, Azza Shoaibi, Gowtham A Rao, Anna Ostropolets, Peter R Rijnbeek, Erica A Voss, Talita Duarte-Salles, Juan Manuel Ramírez-Anguita, Miguel A Mayer, Filip Maljković, Spiros Denaxas, Fredrik Nyberg, Vaclav Papez, Anthony G Sena, Thamir M Alshammari, Lana Y H Lai, Kevin Haynes, Marc A Suchard, George Hripcsak, Patrick B Ryan","doi":"10.1093/jamiaopen/ooad096","DOIUrl":"10.1093/jamiaopen/ooad096","url":null,"abstract":"<p><strong>Objective: </strong>Developing accurate phenotype definitions is critical in obtaining reliable and reproducible background rates in safety research. This study aims to illustrate the differences in background incidence rates by comparing definitions for a given outcome.</p><p><strong>Materials and methods: </strong>We used 16 data sources to systematically generate and evaluate outcomes for 13 adverse events and their overall background rates. We examined the effect of different modifications (inpatient setting, standardization of code set, and code set changes) to the computable phenotype on background incidence rates.</p><p><strong>Results: </strong>Rate ratios (RRs) of the incidence rates from each computable phenotype definition varied across outcomes, with inpatient restriction showing the highest variation from 1 to 11.93. Standardization of code set RRs ranges from 1 to 1.64, and code set changes range from 1 to 2.52.</p><p><strong>Discussion: </strong>The modification that has the highest impact is requiring inpatient place of service, leading to at least a 2-fold higher incidence rate in the base definition. Standardization showed almost no change when using source code variations. The strength of the effect in the inpatient restriction is highly dependent on the outcome. Changing definitions from broad to narrow showed the most variability by age/gender/database across phenotypes and less than a 2-fold increase in rate compared to the base definition.</p><p><strong>Conclusion: </strong>Characterization of outcomes across a network of databases yields insights into sensitivity and specificity trade-offs when definitions are altered. Outcomes should be thoroughly evaluated prior to use for background rates for their plausibility for use across a global network.</p>","PeriodicalId":36278,"journal":{"name":"JAMIA Open","volume":"6 4","pages":"ooad096"},"PeriodicalIF":2.1,"publicationDate":"2023-11-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10662662/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138463160","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}