Pub Date : 2024-11-13DOI: 10.1016/j.ijmedinf.2024.105698
Dong Hoon Jang , Ji Won Heo , Kyu Hong Lee , Ro Woon Lee , Tae Ran Ahn , Hyun Gyu Lee
Introduction
Ultrasound equipment provides real-time visualization of internal organs, essential for early disease detection and diagnosis. However, poor-quality ultrasound images can compromise diagnostic accuracy and increase the risk of misdiagnosis. Quality assessments are often subjective, relying on the evaluator's experience and interpretation, which can vary.
Methods
This study introduces a two-stage deep learning framework designed to objectively assess ultrasound image quality using phantom data across three key parameters: ‘Dead zone’, ‘Axial/lateral resolution’, and ‘Gray scale and dynamic range’. Stage 1 automatically extracts regions of interest for each parameter, while Stage 2 employs detection or classification models to evaluate image quality within these regions. To generate an overall equipment quality score, a logistic regression model combines the weighted results from each parameter.
Results
The classification model demonstrated high performance across datasets, achieving AUC scores of 98.6% for ‘Dead zone’, 87.7% for ‘Axial/lateral resolution’, and 96.0% for ‘Gray scale and dynamic range’. Further analysis using guideline-compliant images of individual devices showed AUC scores of 98.2%, 92.8%, and 100%, respectively. These findings highlight deep learning's potential for quantitative and objective assessments of ultrasound image quality. Ultimately, this framework provides a streamlined approach to quality management, enabling consistent quality control and efficient scoring-based evaluation of ultrasound equipment.
{"title":"Deep learning-driven ultrasound equipment quality assessment with ATS-539 phantom data","authors":"Dong Hoon Jang , Ji Won Heo , Kyu Hong Lee , Ro Woon Lee , Tae Ran Ahn , Hyun Gyu Lee","doi":"10.1016/j.ijmedinf.2024.105698","DOIUrl":"10.1016/j.ijmedinf.2024.105698","url":null,"abstract":"<div><h3>Introduction</h3><div>Ultrasound equipment provides real-time visualization of internal organs, essential for early disease detection and diagnosis. However, poor-quality ultrasound images can compromise diagnostic accuracy and increase the risk of misdiagnosis. Quality assessments are often subjective, relying on the evaluator's experience and interpretation, which can vary.</div></div><div><h3>Methods</h3><div>This study introduces a two-stage deep learning framework designed to objectively assess ultrasound image quality using phantom data across three key parameters: ‘Dead zone’, ‘Axial/lateral resolution’, and ‘Gray scale and dynamic range’. Stage 1 automatically extracts regions of interest for each parameter, while Stage 2 employs detection or classification models to evaluate image quality within these regions. To generate an overall equipment quality score, a logistic regression model combines the weighted results from each parameter.</div></div><div><h3>Results</h3><div>The classification model demonstrated high performance across datasets, achieving AUC scores of 98.6% for ‘Dead zone’, 87.7% for ‘Axial/lateral resolution’, and 96.0% for ‘Gray scale and dynamic range’. Further analysis using guideline-compliant images of individual devices showed AUC scores of 98.2%, 92.8%, and 100%, respectively. These findings highlight deep learning's potential for quantitative and objective assessments of ultrasound image quality. Ultimately, this framework provides a streamlined approach to quality management, enabling consistent quality control and efficient scoring-based evaluation of ultrasound equipment.</div></div>","PeriodicalId":54950,"journal":{"name":"International Journal of Medical Informatics","volume":"193 ","pages":"Article 105698"},"PeriodicalIF":3.7,"publicationDate":"2024-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142633144","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-12DOI: 10.1016/j.ijmedinf.2024.105693
Iiris Hörhammer , Johanna Suvanto , Maarit Kinnunen , Sari Kujala
Background
Remote services provided via telephone or the internet have become an essential part of mental health provision. Alongside services involving healthcare personnel (HCP), self-guided digital services hold great promise for improved self-management and health outcomes without increasing the burden on HCP. Therefore, better understanding of patients’ use and experienced benefits of these services are needed. This study investigated how health confidence and sociodemographic background are associated with mental health patients’ experiences of self-guided digital services.
Methods
This cross-sectional survey study was performed in 2022 at a Finnish Mental Health and Substance Abuse Services (MHSAS) unit of a regional public service provider that serves a population of about 163 000 people. All patients who had visited the unit up to 6 months before the study were invited to respond to an online survey on their experiences with the remote MHSAS. We report the average subjective usefulness of telephone, guided digital and self-guided digital services. Regression models were fitted to study the associations of patient characteristics with use of any digital service, and with experienced usefulness of self-guided digital services.
Findings
The respondents (n = 438) rated the usefulness of telephone, guided digital and self-guided digital services similarly (4.0/5.0, 3.9/5.0, and 3.9/5.0, respectively). Health confidence was associated with not using digital services at all as well as with high perceived usefulness of self-guided services. While elderly patients were more likely to avoid using digital services, age was not associated with experienced usefulness of self-guided digital services. No association between unemployment status and experiences of digital services was found.
Conclusions
Different types of remote services are perceived as beneficial by mental health patients. To ensure effectiveness and equity, patients’ health confidence should be considered when directing them to self-guided services. Elderly mental health patients who use digital services are equally able as younger patients to benefit from self-guided services.
{"title":"Usefulness of self-guided digital services among mental health patients: The role of health confidence and sociodemographic characteristics","authors":"Iiris Hörhammer , Johanna Suvanto , Maarit Kinnunen , Sari Kujala","doi":"10.1016/j.ijmedinf.2024.105693","DOIUrl":"10.1016/j.ijmedinf.2024.105693","url":null,"abstract":"<div><h3>Background</h3><div>Remote services provided via telephone or the internet have become an essential part of mental health provision. Alongside services involving healthcare personnel (HCP), self-guided digital services hold great promise for improved self-management and health outcomes without increasing the burden on HCP. Therefore, better understanding of patients’ use and experienced benefits of these services are needed. This study investigated how health confidence and sociodemographic background are associated with mental health patients’ experiences of self-guided digital services.</div></div><div><h3>Methods</h3><div>This cross-sectional survey study was performed in 2022 at a Finnish Mental Health and Substance Abuse Services (MHSAS) unit of a regional public service provider that serves a population of about 163<!--> <!-->000 people. All patients who had visited the unit up to 6 months before the study were invited to respond to an online survey on their experiences with the remote MHSAS. We report the average subjective usefulness of telephone, guided digital and self-guided digital services. Regression models were fitted to study the associations of patient characteristics with use of any digital service, and with experienced usefulness of self-guided digital services.</div></div><div><h3>Findings</h3><div>The respondents (n = 438) rated the usefulness of telephone, guided digital and self-guided digital services similarly (4.0/5.0, 3.9/5.0, and 3.9/5.0, respectively). Health confidence was associated with not using digital services at all as well as with high perceived usefulness of self-guided services. While elderly patients were more likely to avoid using digital services, age was not associated with experienced usefulness of self-guided digital services. No association between unemployment status and experiences of digital services was found.</div></div><div><h3>Conclusions</h3><div>Different types of remote services are perceived as beneficial by mental health patients. To ensure effectiveness and equity, patients’ health confidence should be considered when directing them to self-guided services. Elderly mental health patients who use digital services are equally able as younger patients to benefit from self-guided services.</div></div>","PeriodicalId":54950,"journal":{"name":"International Journal of Medical Informatics","volume":"194 ","pages":"Article 105693"},"PeriodicalIF":3.7,"publicationDate":"2024-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142656282","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-12DOI: 10.1016/j.ijmedinf.2024.105679
Sivan Bershan , Andreas Meisel , Philipp Mergenthaler
Objective
Myasthenic crisis (MC) is a critical progression of Myasthenia gravis (MG), requiring intensive care treatment and invasive therapies. Classifying patients at high-risk for MC facilitates treatment decisions such as changes in medication or the need for mechanical ventilation and helps prevent disease progression by decreasing treatment-induced stress on the patient. Here, we investigated whether it is possible to reliably classify MG patients into groups at low or high risk of MC based entirely on routine medical data using explainable machine learning (ML).
Methods
In this single-center pseudo-prospective cohort study, we investigated the precision of ML models trained with real-world routine clinical data to identify MG patients at risk for MC, and identified explainable distinctive features for the groups. 51 MG patients, including 13 MC, were used for model training based on real-world clinical data available from the hospital management system. Patients were classified to high or low risk for MC using Lasso regression or random forest ML models.
Results
The mean cross-validated AUC classifying MG patients as high or low risk for MC based on simple or compound features derived from real-world clinical data showed a predictive accuracy of 68.8% for a regularized Lasso regression and 76.5% for a random forest model. Studying feature importance across 5100 model runs identified explainable features to distinguish MG patients at high or low risk for MC. Feature importance scores suggested that multimorbidity may play a role in risk classification.
Conclusion
This study establishes feasibility and proof-of-concept for risk classification of MC based on real-world routine clinical data using ML with explainable features and variance control at the point of care. Future research on ML-based prediction of MC should include multi-center, multinational data collection, more in-depth data per patient, more patients, and an attention-based ML model to include free-text.
目的：肌无力危象（MC）是重症肌无力症（MG）的一个重要进展，需要重症监护治疗和侵入性疗法。对MC高危患者进行分类有助于做出治疗决定，如更换药物或是否需要机械通气，并通过减少治疗对患者造成的压力来预防疾病进展。在此，我们利用可解释的机器学习（ML）研究了是否有可能完全根据常规医疗数据将 MG 患者可靠地分为 MC 低风险或高风险组：在这项单中心伪前瞻性队列研究中，我们研究了使用真实世界常规临床数据训练的ML模型识别MG患者MC风险的精确度，并确定了各组可解释的显著特征。根据医院管理系统提供的真实世界临床数据，对 51 名 MG 患者（包括 13 名 MC）进行了模型训练。使用 Lasso 回归或随机森林 ML 模型将患者划分为 MC 高风险或低风险：根据真实世界临床数据中的简单或复合特征将 MG 患者划分为 MC 高风险或低风险的交叉验证 AUC 平均值显示，正则化 Lasso 回归的预测准确率为 68.8%，随机森林模型的预测准确率为 76.5%。通过对 5100 次模型运行的特征重要性进行研究，确定了可用于区分 MC 高风险或低风险 MG 患者的可解释特征。特征重要性得分表明，多病性可能在风险分类中发挥作用：本研究基于真实世界的常规临床数据，利用具有可解释特征的 ML 和护理点的方差控制，建立了 MC 风险分类的可行性和概念验证。基于 ML 的 MC 预测的未来研究应包括多中心、跨国数据收集、每个患者更深入的数据、更多的患者以及基于注意力的 ML 模型（包括自由文本）。
{"title":"Data-driven explainable machine learning for personalized risk classification of myasthenic crisis","authors":"Sivan Bershan , Andreas Meisel , Philipp Mergenthaler","doi":"10.1016/j.ijmedinf.2024.105679","DOIUrl":"10.1016/j.ijmedinf.2024.105679","url":null,"abstract":"<div><h3>Objective</h3><div>Myasthenic crisis (MC) is a critical progression of Myasthenia gravis (MG), requiring intensive care treatment and invasive therapies. Classifying patients at high-risk for MC facilitates treatment decisions such as changes in medication or the need for mechanical ventilation and helps prevent disease progression by decreasing treatment-induced stress on the patient. Here, we investigated whether it is possible to reliably classify MG patients into groups at low or high risk of MC based entirely on routine medical data using explainable machine learning (ML).</div></div><div><h3>Methods</h3><div>In this single-center pseudo-prospective cohort study, we investigated the precision of ML models trained with real-world routine clinical data to identify MG patients at risk for MC, and identified explainable distinctive features for the groups. 51 MG patients, including 13 MC, were used for model training based on real-world clinical data available from the hospital management system. Patients were classified to high or low risk for MC using Lasso regression or random forest ML models.</div></div><div><h3>Results</h3><div>The mean cross-validated AUC classifying MG patients as high or low risk for MC based on simple or compound features derived from real-world clinical data showed a predictive accuracy of 68.8% for a regularized Lasso regression and 76.5% for a random forest model. Studying feature importance across 5100 model runs identified explainable features to distinguish MG patients at high or low risk for MC. Feature importance scores suggested that multimorbidity may play a role in risk classification.</div></div><div><h3>Conclusion</h3><div>This study establishes feasibility and proof-of-concept for risk classification of MC based on real-world routine clinical data using ML with explainable features and variance control at the point of care. Future research on ML-based prediction of MC should include multi-center, multinational data collection, more in-depth data per patient, more patients, and an attention-based ML model to include free-text.</div></div>","PeriodicalId":54950,"journal":{"name":"International Journal of Medical Informatics","volume":"194 ","pages":"Article 105679"},"PeriodicalIF":3.7,"publicationDate":"2024-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142683779","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-10DOI: 10.1016/j.ijmedinf.2024.105687
Roberto Tornero Costa , Keyrellous Adib , Nagui Salama , Stefania Davia , Antonio Martínez Millana , Vicente Traver , Karapet Davtyan
<div><div>Electronic health record (EHR) systems are powerful tools that enhance healthcare quality. They improve efficiency, enable data exchange, and ensure authorized access to patient information. In 2022, the World Health Organization Regional Office for Europe (WHO EURO) conducted a survey to assess the digital health capabilities of the 53 Member States. This article provides a sub-regional analysis of the status of EHR systems and major barriers to their implementation, their readiness for information sharing, and the access and re-use of EHR data.</div><div>Generally, EHR implementation and national data exchange are at an advanced stage in the region, though achievements and challenges vary across subregions. While more Member States in the Eastern, Western, and Southern Europe subregions reported having centralized national EHR systems, the situation is more diverse in Northern Europe and the Asian subregions, where both centralized and decentralized EHR systems are in use. Significant barriers to EHR implementation, including funding, technical capacity, competing priorities, and lack of interoperability standards are frequently cited, while others like demand, knowledge or acceptance challenges are not reported as significant. Significant barriers were reported the most by the Central Asia subregion, while barriers had least significance in Western Europe. Five out of the six subregions reported a wide adoption of national strategies and have dedicated agencies to ensure interoperability and secure data exchange. However, only 29 Member States have established legal requirements for healthcare providers to adopt EHR systems that conform to national standards for both clinical terminology and electronic messaging, with this being most notable in Western, Eastern, and Northern Europe, and the lowest percentage of Member States in Central Asia. All Member States of the six sub regions have passed privacy and data protection legislation. The use of EHR data is widely regulated, with only five remaining Member States of WHO Europe to develop EHR legislation distributed across subregions (Southern Europe, Western and Central Asia).</div><div>Looking ahead, Member States are encouraged to define national legislation governing EHR systems and their use, while ensuring the interconnectivity of the local and regional EHR systems. Sustainable funding should be allocated to the development and maintenance of these systems. Efforts should also focus on creating comprehensive roadmaps for the full implementation of health data standards, addressing interoperability at local and regional levels, and developing quality management systems for testing and certification. Additionally, monitoring and evaluation should be conducted to assess whether EHRs are contributing to national health objectives. Finally, engaging patients and intersectoral partners will be key to developing a more patient-centered approach, ensuring that EHR systems meet patient ne
{"title":"Electronic health records and data exchange in the WHO European region: A subregional analysis of achievements, challenges, and prospects","authors":"Roberto Tornero Costa , Keyrellous Adib , Nagui Salama , Stefania Davia , Antonio Martínez Millana , Vicente Traver , Karapet Davtyan","doi":"10.1016/j.ijmedinf.2024.105687","DOIUrl":"10.1016/j.ijmedinf.2024.105687","url":null,"abstract":"<div><div>Electronic health record (EHR) systems are powerful tools that enhance healthcare quality. They improve efficiency, enable data exchange, and ensure authorized access to patient information. In 2022, the World Health Organization Regional Office for Europe (WHO EURO) conducted a survey to assess the digital health capabilities of the 53 Member States. This article provides a sub-regional analysis of the status of EHR systems and major barriers to their implementation, their readiness for information sharing, and the access and re-use of EHR data.</div><div>Generally, EHR implementation and national data exchange are at an advanced stage in the region, though achievements and challenges vary across subregions. While more Member States in the Eastern, Western, and Southern Europe subregions reported having centralized national EHR systems, the situation is more diverse in Northern Europe and the Asian subregions, where both centralized and decentralized EHR systems are in use. Significant barriers to EHR implementation, including funding, technical capacity, competing priorities, and lack of interoperability standards are frequently cited, while others like demand, knowledge or acceptance challenges are not reported as significant. Significant barriers were reported the most by the Central Asia subregion, while barriers had least significance in Western Europe. Five out of the six subregions reported a wide adoption of national strategies and have dedicated agencies to ensure interoperability and secure data exchange. However, only 29 Member States have established legal requirements for healthcare providers to adopt EHR systems that conform to national standards for both clinical terminology and electronic messaging, with this being most notable in Western, Eastern, and Northern Europe, and the lowest percentage of Member States in Central Asia. All Member States of the six sub regions have passed privacy and data protection legislation. The use of EHR data is widely regulated, with only five remaining Member States of WHO Europe to develop EHR legislation distributed across subregions (Southern Europe, Western and Central Asia).</div><div>Looking ahead, Member States are encouraged to define national legislation governing EHR systems and their use, while ensuring the interconnectivity of the local and regional EHR systems. Sustainable funding should be allocated to the development and maintenance of these systems. Efforts should also focus on creating comprehensive roadmaps for the full implementation of health data standards, addressing interoperability at local and regional levels, and developing quality management systems for testing and certification. Additionally, monitoring and evaluation should be conducted to assess whether EHRs are contributing to national health objectives. Finally, engaging patients and intersectoral partners will be key to developing a more patient-centered approach, ensuring that EHR systems meet patient ne","PeriodicalId":54950,"journal":{"name":"International Journal of Medical Informatics","volume":"194 ","pages":"Article 105687"},"PeriodicalIF":3.7,"publicationDate":"2024-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142655800","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-10DOI: 10.1016/j.ijmedinf.2024.105700
Diana Shamsutdinova , Daniel Stamate , Daniel Stahl
Background
Accurate and interpretable models are essential for clinical decision-making, where predictions can directly impact patient care. Machine learning (ML) survival methods can handle complex multidimensional data and achieve high accuracy but require post-hoc explanations. Traditional models such as the Cox Proportional Hazards Model (Cox-PH) are less flexible, but fast, stable, and intrinsically transparent. Moreover, ML does not always outperform Cox-PH in clinical settings, warranting a diligent model validation. We aimed to develop a set of R functions to help explore the limits of Cox-PH compared to the tree-based and deep learning survival models for clinical prediction modelling, employing ensemble learning and nested cross-validation.
Methods
We developed a set of R functions, publicly available as the package “survcompare”. It supports Cox-PH and Cox-Lasso, and Survival Random Forest (SRF) and DeepHit are the ML alternatives, along with the ensemble methods integrating Cox-PH with SRF or DeepHit designed to isolate the marginal value of ML. The package performs a repeated nested cross-validation and tests for statistical significance of the ML’s superiority using the survival-specific performance metrics, the concordance index, time-dependent AUC-ROC and calibration slope.
To get practical insights, we applied this methodology to clinical and simulated datasets with varying complexities and sizes.
Results
In simulated data with non-linearities or interactions, ML models outperformed Cox-PH at sample sizes ≥ 500. ML superiority was also observed in imaging and high-dimensional clinical data. However, for tabular clinical data, the performance gains of ML were minimal; in some cases, regularised Cox-Lasso recovered much of the ML’s performance advantage with significantly faster computations. Ensemble methods combining Cox-PH and ML predictions were instrumental in quantifying Cox-PH’s limits and improving ML calibration. Traditional models like Cox-PH or Cox-Lasso should not be overlooked while developing clinical predictive models from tabular data or data of limited size.
Conclusion
Our package offers researchers a framework and practical tool for evaluating the accuracy-interpretability trade-off, helping make informed decisions about model selection.
背景：准确且可解释的模型对于临床决策至关重要，因为预测会直接影响到患者护理。机器学习（ML）生存方法可以处理复杂的多维数据并获得高准确性，但需要事后解释。考克斯比例危害模型（Cox-PH）等传统模型灵活性较差，但速度快、稳定性好，而且本质上是透明的。此外，在临床环境中，ML 并不总是优于 Cox-PH，因此需要对模型进行认真的验证。我们的目标是开发一套 R 函数，利用集合学习和嵌套交叉验证，帮助探索 Cox-PH 与基于树和深度学习的生存模型相比在临床预测建模方面的局限性：我们开发了一套 R 函数，作为 "survcompare "软件包公开发布。它支持 Cox-PH 和 Cox-Lasso，生存随机森林（SRF）和 DeepHit 是 ML 的替代方法，以及将 Cox-PH 与 SRF 或 DeepHit 整合在一起的集合方法，旨在分离 ML 的边际价值。该软件包执行重复嵌套交叉验证，并使用生存特定性能指标、一致性指数、随时间变化的 AUC-ROC 和校准斜率检验 ML 优越性的统计显著性。为了获得实用的见解，我们将这种方法应用于具有不同复杂性和规模的临床和模拟数据集：结果：在具有非线性或交互作用的模拟数据中，当样本量≥ 500 时，ML 模型优于 Cox-PH。在成像和高维临床数据中也观察到了 ML 的优越性。然而，在表格临床数据中，ML 的性能提升微乎其微；在某些情况下，正则化 Cox-Lasso 恢复了 ML 的大部分性能优势，而且计算速度明显更快。结合 Cox-PH 和 ML 预测的集合方法有助于量化 Cox-PH 的局限性并改进 ML 校准。在利用表格数据或规模有限的数据开发临床预测模型时，不应忽视 Cox-PH 或 Cox-Lasso 等传统模型：我们的软件包为研究人员提供了评估准确性-可解释性权衡的框架和实用工具，有助于在模型选择方面做出明智的决策。
{"title":"Balancing accuracy and Interpretability: An R package assessing complex relationships beyond the Cox model and applications to clinical prediction","authors":"Diana Shamsutdinova , Daniel Stamate , Daniel Stahl","doi":"10.1016/j.ijmedinf.2024.105700","DOIUrl":"10.1016/j.ijmedinf.2024.105700","url":null,"abstract":"<div><h3>Background</h3><div>Accurate and interpretable models are essential for clinical decision-making, where predictions can directly impact patient care. Machine learning (ML) survival methods can handle complex multidimensional data and achieve high accuracy but require post-hoc explanations. Traditional models such as the Cox Proportional Hazards Model (Cox-PH) are less flexible, but fast, stable, and intrinsically transparent. Moreover, ML does not always outperform Cox-PH in clinical settings, warranting a diligent model validation. We aimed to develop a set of R functions to help explore the limits of Cox-PH compared to the tree-based and deep learning survival models for clinical prediction modelling, employing ensemble learning and nested cross-validation.</div></div><div><h3>Methods</h3><div>We developed a set of R functions, publicly available as the package “survcompare”. It supports Cox-PH and Cox-Lasso, and Survival Random Forest (SRF) and DeepHit are the ML alternatives, along with the ensemble methods integrating Cox-PH with SRF or DeepHit designed to isolate the marginal value of ML. The package performs a repeated nested cross-validation and tests for statistical significance of the ML’s superiority using the survival-specific performance metrics, the concordance index, time-dependent AUC-ROC and calibration slope.</div><div>To get practical insights, we applied this methodology to clinical and simulated datasets with varying complexities and sizes.</div></div><div><h3>Results</h3><div>In simulated data with non-linearities or interactions, ML models outperformed Cox-PH at sample sizes ≥ 500. ML superiority was also observed in imaging and high-dimensional clinical data. However, for tabular clinical data, the performance gains of ML were minimal; in some cases, regularised Cox-Lasso recovered much of the ML’s performance advantage with significantly faster computations. Ensemble methods combining Cox-PH and ML predictions were instrumental in quantifying Cox-PH’s limits and improving ML calibration. Traditional models like Cox-PH or Cox-Lasso should not be overlooked while developing clinical predictive models from tabular data or data of limited size.</div></div><div><h3>Conclusion</h3><div>Our package offers researchers a framework and practical tool for evaluating the accuracy-interpretability trade-off, helping make informed decisions about model selection.</div></div>","PeriodicalId":54950,"journal":{"name":"International Journal of Medical Informatics","volume":"194 ","pages":"Article 105700"},"PeriodicalIF":3.7,"publicationDate":"2024-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142640312","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-10DOI: 10.1016/j.ijmedinf.2024.105699
Carlos M. Mejía-Granda, José L. Fernández-Alemán, Juan M. Carrillo de Gea, José A. García-Berná
Objective
This article deals with the complex process of obtaining security requirements for e-Health applications. It introduces a tailored audit and validation methodology particularly designed for e-Health applications. Additionally, it presents a comprehensive security catalog derived from primary sources such as law, guides, standards, best practices, and a systematic literature review. This catalog is characterized by its continuous improvement, clarity, completeness, consistency, verifiability, modifiability, and traceability.
Methods
The authors reviewed electronic health security literature and gathered primary sources of law, guides, standards, and best practices. They organized the catalog according to the ISO/IEC/IEEE 29148:2018 standard and proposed a methodology to ensure its reusability. Moreover, the authors proposed SEC-AM as an audit method. The applicability of the catalog was validated through the audit method, which was conducted on a prominent medical application, OpenEMR.
Results
The proposed method and validation for auditing e-Health Applications through the catalog provided a comprehensive framework for developing or evaluating new applications. Through the audit of OpenEMR, several security vulnerabilities were identified, such as DDOs, XSS, JSONi, and CMDi, resulting in a “Secure” classification of OpenEMR with a compliance rate of 66.97%.
Conclusion
The study demonstrates the proposed catalog’s feasibility and effectiveness in enhancing health software security. The authors suggest continuous improvement by incorporating new regulations, knowledge from additional sources, and addressing emerging zero-day vulnerabilities. This approach is crucial for providing practical, safe, and quality medical care amidst increasing cyber threats in the healthcare industry.
{"title":"A method and validation for auditing e-Health applications based on reusable software security requirements specifications","authors":"Carlos M. Mejía-Granda, José L. Fernández-Alemán, Juan M. Carrillo de Gea, José A. García-Berná","doi":"10.1016/j.ijmedinf.2024.105699","DOIUrl":"10.1016/j.ijmedinf.2024.105699","url":null,"abstract":"<div><h3>Objective</h3><div>This article deals with the complex process of obtaining security requirements for e-Health applications. It introduces a tailored audit and validation methodology particularly designed for e-Health applications. Additionally, it presents a comprehensive security catalog derived from primary sources such as law, guides, standards, best practices, and a systematic literature review. This catalog is characterized by its continuous improvement, clarity, completeness, consistency, verifiability, modifiability, and traceability.</div></div><div><h3>Methods</h3><div>The authors reviewed electronic health security literature and gathered primary sources of law, guides, standards, and best practices. They organized the catalog according to the ISO/IEC/IEEE 29148:2018 standard and proposed a methodology to ensure its reusability. Moreover, the authors proposed SEC-AM as an audit method. The applicability of the catalog was validated through the audit method, which was conducted on a prominent medical application, OpenEMR.</div></div><div><h3>Results</h3><div>The proposed method and validation for auditing e-Health Applications through the catalog provided a comprehensive framework for developing or evaluating new applications. Through the audit of OpenEMR, several security vulnerabilities were identified, such as DDOs, XSS, JSONi, and CMDi, resulting in a “Secure” classification of OpenEMR with a compliance rate of 66.97%.</div></div><div><h3>Conclusion</h3><div>The study demonstrates the proposed catalog’s feasibility and effectiveness in enhancing health software security. The authors suggest continuous improvement by incorporating new regulations, knowledge from additional sources, and addressing emerging zero-day vulnerabilities. This approach is crucial for providing practical, safe, and quality medical care amidst increasing cyber threats in the healthcare industry.</div></div>","PeriodicalId":54950,"journal":{"name":"International Journal of Medical Informatics","volume":"194 ","pages":"Article 105699"},"PeriodicalIF":3.7,"publicationDate":"2024-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142704836","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-09DOI: 10.1016/j.ijmedinf.2024.105684
Florian Bösch , Stina Schild-Suhren , Elif Yilmaz , Michael Ghadimi , Athanasios Karampalis , Nikolaus Börner , Markus Bo Schoenberg
Background
The assessment of clinical outcome quality, particularly in surgery, is crucial for healthcare improvement. Traditional cross-sectional analyses often fall short in timely and systematic identification of clinical quality issues. This study explores the efficacy of machine learning adjusted sequential CUSUM (Cumulative Sum) analyses in monitoring post-surgical mortality.
Material and methods
Utilizing the Global Open Source Severity of Illness Score (GOSSIS) dataset involving 91,714 patient records from 147 hospitals, this study involved the development of a machine learning model for mortality using a modified LightGBM algorithm. With this, sequential and cross sectional quality monitoring was simulated and compared.
Results
The modified LightGBM model demonstrated superior predictive accuracy (ROC AUC of 0.88). Simulations revealed that the AI risk-adjusted CUSUM required fewer patient outcome alterations to detect atypical trends compared to standard methods.
Conclusion
The AI risk-adjusted CUSUM analysis represents a significant advancement in monitoring clinical outcome quality in healthcare, especially in surgery. Its ability to detect minor discrepancies in mortality rates with greater sensitivity and specificity positions it as a valuable tool for healthcare providers. This approach could lead to earlier interventions and improved patient care.
{"title":"Machine learning adjusted sequential CUSUM-analyses are superior to cross-sectional analysis of excess mortality after surgery","authors":"Florian Bösch , Stina Schild-Suhren , Elif Yilmaz , Michael Ghadimi , Athanasios Karampalis , Nikolaus Börner , Markus Bo Schoenberg","doi":"10.1016/j.ijmedinf.2024.105684","DOIUrl":"10.1016/j.ijmedinf.2024.105684","url":null,"abstract":"<div><h3>Background</h3><div>The assessment of clinical outcome quality, particularly in surgery, is crucial for healthcare improvement. Traditional cross-sectional analyses often fall short in timely and systematic identification of clinical quality issues. This study explores the efficacy of machine learning adjusted sequential CUSUM (Cumulative Sum) analyses in monitoring post-surgical mortality.</div></div><div><h3>Material and methods</h3><div>Utilizing the Global Open Source Severity of Illness Score (GOSSIS) dataset involving 91,714 patient records from 147 hospitals, this study involved the development of a machine learning model for mortality using a modified LightGBM algorithm. With this, sequential and cross sectional quality monitoring was simulated and compared.</div></div><div><h3>Results</h3><div>The modified LightGBM model demonstrated superior predictive accuracy (ROC AUC of 0.88). Simulations revealed that the AI risk-adjusted CUSUM required fewer patient outcome alterations to detect atypical trends compared to standard methods.</div></div><div><h3>Conclusion</h3><div>The AI risk-adjusted CUSUM analysis represents a significant advancement in monitoring clinical outcome quality in healthcare, especially in surgery. Its ability to detect minor discrepancies in mortality rates with greater sensitivity and specificity positions it as a valuable tool for healthcare providers. This approach could lead to earlier interventions and improved patient care.</div></div>","PeriodicalId":54950,"journal":{"name":"International Journal of Medical Informatics","volume":"193 ","pages":"Article 105684"},"PeriodicalIF":3.7,"publicationDate":"2024-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142640309","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-09DOI: 10.1016/j.ijmedinf.2024.105681
Evangelia Kyrimi , Somayyeh Mossadegh , Jared M. Wohlgemut , Rebecca S. Stoner , Nigel R.M. Tai , William Marsh
Background
Healthcare governance (HG) is a quality assurance processes that aims to maintain and improve clinical practice. Clinical decisions are routinely reviewed after the outcome is known to learn lessons for the future. When the outcome is positive, then practice is praised, but when practice is suboptimal, the area for improvement is highlighted. This process requires counterfactual reasoning, where we predict what would have happened given both what happened and the possible different decisions. Causal models that capture the mechanisms that generate events can support counterfactual reasoning.
Objective
This study is an initial attempt to show how counterfactual reasoning with causal Bayesian networks (CBNs) can be used as a HG tool to assess what would have happened if treatments other than those occurred had been selected.
Methods
Motivated by the Defence Medical Services (DMS) mortality and morbidity (M&M) review meeting, in this paper we (1) extended the use of counterfactual reasoning in CBNs to review decisions, where the alternative treatment strategies and its effect belong to different stages of care, (2) placed counterfactual reasoning in a specific clinical context to examine how it can be used as a HG tool.
Results
Using three realistic examples, we demonstrated how the proposed counterfactual reasoning can be used to assist the DMS M&M review meetings.
Conclusions
Useful lessons can be learned by assessing decisions after they are made. M&M review meetings are fruitful ground for counterfactual reasoning. The use of a clinical decision support tool that can assist clinicians in assessing counterfactual probabilities will be beneficial.
{"title":"Counterfactual reasoning using causal Bayesian networks as a healthcare governance tool","authors":"Evangelia Kyrimi , Somayyeh Mossadegh , Jared M. Wohlgemut , Rebecca S. Stoner , Nigel R.M. Tai , William Marsh","doi":"10.1016/j.ijmedinf.2024.105681","DOIUrl":"10.1016/j.ijmedinf.2024.105681","url":null,"abstract":"<div><h3>Background</h3><div>Healthcare governance (HG) is a quality assurance processes that aims to maintain and improve clinical practice. Clinical decisions are routinely reviewed after the outcome is known to learn lessons for the future. When the outcome is positive, then practice is praised, but when practice is suboptimal, the area for improvement is highlighted. This process requires counterfactual reasoning, where we predict what would have happened given both what happened and the possible different decisions. Causal models that capture the mechanisms that generate events can support counterfactual reasoning.</div></div><div><h3>Objective</h3><div>This study is an initial attempt to show how counterfactual reasoning with causal Bayesian networks (CBNs) can be used as a HG tool to assess what would have happened if treatments other than those occurred had been selected.</div></div><div><h3>Methods</h3><div>Motivated by the Defence Medical Services (DMS) mortality and morbidity (M&M) review meeting, in this paper we (1) extended the use of counterfactual reasoning in CBNs to review decisions, where the alternative treatment strategies and its effect belong to different stages of care, (2) placed counterfactual reasoning in a specific clinical context to examine how it can be used as a HG tool.</div></div><div><h3>Results</h3><div>Using three realistic examples, we demonstrated how the proposed counterfactual reasoning can be used to assist the DMS M&M review meetings.</div></div><div><h3>Conclusions</h3><div>Useful lessons can be learned by assessing decisions after they are made. M&M review meetings are fruitful ground for counterfactual reasoning. The use of a clinical decision support tool that can assist clinicians in assessing counterfactual probabilities will be beneficial.</div></div>","PeriodicalId":54950,"journal":{"name":"International Journal of Medical Informatics","volume":"193 ","pages":"Article 105681"},"PeriodicalIF":3.7,"publicationDate":"2024-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142633138","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-07DOI: 10.1016/j.ijmedinf.2024.105663
Emine Karacan
Background
As artificial intelligence AI-supported applications become integral to web-based information-seeking, assessing their impact on healthy nutrition and weight management during the antenatal period is crucial.
Objective
This study was conducted to evaluate both the quality and semantic similarity of responses created by AI models to the most frequently asked questions about healthy nutrition and weight management during the antenatal period, based on existing clinical knowledge.
Methods
In this study, a cross-sectional assessment design was used to explore data from 3 AI models (GPT-4, MedicalGPT, Med-PaLM). We directed the most frequently asked questions about nutrition during pregnancy, obtained from the American College of Obstetricians and Gynecologists (ACOG) to each model in a new and single session on October 21, 2023, without any prior conversation. Immediately after, instructions were given to the AI models to generate responses to these questions. The responses created by AI models were evaluated using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) scale. Additionally, to assess the semantic similarity between answers to 31 pregnancy nutrition-related frequently asked questions sourced from the ACOG and responses from AI models we evaluated cosine similarity using both WORD2VEC and BioLORD-2023.
Results
Med-PaLM outperformed GPT-4 and MedicalGPT in response quality (mean = 3.93), demonstrating superior clinical accuracy over both GPT-4 (p = 0.016) and MedicalGPT (p = 0.001). GPT-4 had higher quality than MedicalGPT (p = 0.027).
The semantic similarity between ACOG and Med-PaLM is higher with WORD2VEC (0.92) compared to BioLORD-2023 (0.81), showing a difference of +0.11. The similarity scores for ACOG–MedicalGPT and ACOG–GPT-4 are similar across both models, with minimal differences of −0.01. Overall, WORD2VEC has a slightly higher average similarity (0.82) than BioLORD-2023 (0.79), with a difference of +0.03.
Conclusions
Despite the superior performance of Med-PaLM, there is a need for further evidence-based research and improvement in the integration of AI in healthcare due to varying AI model performances.
{"title":"Healthy nutrition and weight management for a positive pregnancy experience in the antenatal period: Comparison of responses from artificial intelligence models on nutrition during pregnancy","authors":"Emine Karacan","doi":"10.1016/j.ijmedinf.2024.105663","DOIUrl":"10.1016/j.ijmedinf.2024.105663","url":null,"abstract":"<div><h3>Background</h3><div>As artificial intelligence AI-supported applications become integral to web-based information-seeking, assessing their impact on healthy nutrition and weight management during the antenatal period is crucial.</div></div><div><h3>Objective</h3><div>This study was conducted to evaluate both the quality and semantic similarity of responses created by AI models to the most frequently asked questions about healthy nutrition and weight management during the antenatal period, based on existing clinical knowledge.</div></div><div><h3>Methods</h3><div>In this study, a cross-sectional assessment design was used to explore data from 3 AI models (GPT-4, MedicalGPT, Med-PaLM). We directed the most frequently asked questions about nutrition during pregnancy, obtained from the American College of Obstetricians and Gynecologists (ACOG) to each model in a new and single session on October 21, 2023, without any prior conversation. Immediately after, instructions were given to the AI models to generate responses to these questions. The responses created by AI models were evaluated using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) scale. Additionally, to assess the semantic similarity between answers to 31 pregnancy nutrition-related frequently asked questions sourced from the ACOG and responses from AI models we evaluated cosine similarity using both WORD2VEC and BioLORD-2023.</div></div><div><h3>Results</h3><div>Med-PaLM outperformed GPT-4 and MedicalGPT in response quality (mean = 3.93), demonstrating superior clinical accuracy over both GPT-4 (p = 0.016) and MedicalGPT (p = 0.001). GPT-4 had higher quality than MedicalGPT (p = 0.027).</div><div>The semantic similarity between ACOG and Med-PaLM is higher with WORD2VEC (0.92) compared to BioLORD-2023 (0.81), showing a difference of +0.11. The similarity scores for ACOG–MedicalGPT and ACOG–GPT-4 are similar across both models, with minimal differences of −0.01. Overall, WORD2VEC has a slightly higher average similarity (0.82) than BioLORD-2023 (0.79), with a difference of +0.03.</div></div><div><h3>Conclusions</h3><div>Despite the superior performance of Med-PaLM, there is a need for further evidence-based research and improvement in the integration of AI in healthcare due to varying AI model performances.</div></div>","PeriodicalId":54950,"journal":{"name":"International Journal of Medical Informatics","volume":"193 ","pages":"Article 105663"},"PeriodicalIF":3.7,"publicationDate":"2024-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142633152","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-07DOI: 10.1016/j.ijmedinf.2024.105692
Ta-Chuan Yu , Cheng-Kun Yang , Wei-Han Hsu , Cheng-An Hsu , Hsiao-Chun Wang , Hsin-Jung Hsiao , Hsiao-Ling Chao , Han-Peng Hsieh , Jia-Rong Wu , Yen-Chun Tsai , Yi-Mei Chiang , Poshing Lee , Che-Pin Lin , Ling-Ping Chen , Yung-Chuan Sung , Ya-Yun Yang , Chin-Ling Yu , Chih-Kang Lin , Chia-Pin Kang , Che-Wei Chang , Wen-Chien Chou
Background
Differential counting (DC) of different cell types in bone marrow (BM) aspiration smears is crucial for diagnosing hematological diseases. However, a clinically applicable method for automatic DC has yet to be developed.
Objective
This study developed and validated an artificial intelligence (AI)-based algorithm for identifying and classifying nucleated cells in BM smears.
Methods
In the development phase, a mask region–based convolutional neural network (Mask R-CNN)-based AI model was trained to detect and classify individual BM cells. We used a large data set of expert-annotated images representing a variety of disease categories. The BM slides were stained with Liu’s stain or Wright–Giemsa stain. Consensus meetings were held to ensure experts from different institutes applied consistent criteria in classifying cells. Subsequently, the performance of the AI algorithm in identifying cell images and determining cell ratios was evaluated using a multinational clinical dataset.
Results
The AI model was trained on 542 slides (85.1 % stained with Liu’s stain and 14.9 % with Wright–Giemsa stain) containing 597,222 annotated cells. It achieved an accuracy of 0.94 for the testing dataset containing 26,170 cells. The performance of the AI model was further validated using another multinational real-world dataset (data obtained from three centers in Taiwan and one in the United States) comprising 200,639 cells. The AI model achieved an accuracy of 0.881 in classifying individual cells and demonstrated high precision in classifying blasts (0.927), bands and polymorphonuclear neutrophils (0.955), plasma cells (0.930), and lymphocytes (0.789). When the differential counting percentage of each cell type was assessed, a strong correlation (ρ > 0.8) between the AI and manual methods was observed for most cell categories.
Conclusions
In this study, an AI algorithm was developed and clinically validated using large, multinational datasets. Our algorithm can locate and classify BM cells simultaneously and has potential clinical applicability for automating BM differential counting.
{"title":"A machine-learning-based algorithm for bone marrow cell differential counting","authors":"Ta-Chuan Yu , Cheng-Kun Yang , Wei-Han Hsu , Cheng-An Hsu , Hsiao-Chun Wang , Hsin-Jung Hsiao , Hsiao-Ling Chao , Han-Peng Hsieh , Jia-Rong Wu , Yen-Chun Tsai , Yi-Mei Chiang , Poshing Lee , Che-Pin Lin , Ling-Ping Chen , Yung-Chuan Sung , Ya-Yun Yang , Chin-Ling Yu , Chih-Kang Lin , Chia-Pin Kang , Che-Wei Chang , Wen-Chien Chou","doi":"10.1016/j.ijmedinf.2024.105692","DOIUrl":"10.1016/j.ijmedinf.2024.105692","url":null,"abstract":"<div><h3>Background</h3><div>Differential counting (DC) of different cell types in bone marrow (BM) aspiration smears is crucial for diagnosing hematological diseases. However, a clinically applicable method for automatic DC has yet to be developed.</div></div><div><h3>Objective</h3><div>This study developed and validated an artificial intelligence (AI)-based algorithm for identifying and classifying nucleated cells in BM smears.</div></div><div><h3>Methods</h3><div>In the development phase, a mask region–based convolutional neural network (Mask R-CNN)-based AI model was trained to detect and classify individual BM cells. We used a large data set of expert-annotated images representing a variety of disease categories. The BM slides were stained with Liu’s stain or Wright–Giemsa stain. Consensus meetings were held to ensure experts from different institutes applied consistent criteria in classifying cells. Subsequently, the performance of the AI algorithm in identifying cell images and determining cell ratios was evaluated using a multinational clinical dataset.</div></div><div><h3>Results</h3><div>The AI model was trained on 542 slides (85.1 % stained with Liu’s stain and 14.9 % with Wright–Giemsa stain) containing 597,222 annotated cells. It achieved an accuracy of 0.94 for the testing dataset containing 26,170 cells. The performance of the AI model was further validated using another multinational real-world dataset (data obtained from three centers in Taiwan and one in the United States) comprising 200,639 cells. The AI model achieved an accuracy of 0.881 in classifying individual cells and demonstrated high precision in classifying blasts (0.927), bands and polymorphonuclear neutrophils (0.955), plasma cells (0.930), and lymphocytes (0.789). When the differential counting percentage of each cell type was assessed, a strong correlation (ρ > 0.8) between the AI and manual methods was observed for most cell categories.</div></div><div><h3>Conclusions</h3><div>In this study, an AI algorithm was developed and clinically validated using large, multinational datasets. Our algorithm can locate and classify BM cells simultaneously and has potential clinical applicability for automating BM differential counting.</div></div>","PeriodicalId":54950,"journal":{"name":"International Journal of Medical Informatics","volume":"194 ","pages":"Article 105692"},"PeriodicalIF":3.7,"publicationDate":"2024-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142645226","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号