John Mutersbaugh, Wan-Chun Su, Anjana Bhat, Amir Gandjbakhche
Background: Autism spectrum disorder (ASD) is a prevalent neurodevelopmental condition that can be quite difficult to diagnose due to a lack of objective diagnostic methods in the currently used behavioral assessments. Recent work has shown that children with ASD have a higher incidence of motor control differences. A compilation of studies indicates that between 50% and 88% of the children with ASD have issues with movement control based on standardized motor assessments or parent-reported questionnaires.
Objective: In this study, we assess a variety of deep learning approaches for the classification of ASD, utilizing data collected via inertial measurement unit (IMU) hand tracking during goal-directed arm movements.
Methods: IMU hand tracking data were recorded from 41 school-aged children both with and without an ASD diagnosis to track their arm movements during a reach-to-clean up task. The IMU data were then preprocessed using a moving average and z score normalization to prepare the data for deep learning models. We evaluated the effectiveness of different deep learning models using the preprocessed data and a k-fold validation approach, as well as a patient-separated approach.
Results: The best result was achieved with a convolutional autoencoder combined with long short-term memory layers, reaching an accuracy of 90.21% and an F1-score of 90.02%. Once the convolutional autoencoder+long short-term memory was determined to be the most effective model for this datatype, it was retrained and evaluated with a patient-separated dataset to assess the generalization capability of the model, achieving an accuracy of 91.87% and an F1-score of 93.66%.
Conclusions: Our deep learning approach demonstrates that our models hold potential for facilitating ASD diagnosis in clinical settings. This work validates that there are significant differences between the physical movements of typically developing children and children with ASD, and these differences can be identified by analyzing hand-eye coordination skills. Additionally, we have validated that small-scale models can still achieve a high accuracy and good generalization when classifying medical data, opening the door for future research into diagnostic models that may not require massive amounts of data.
{"title":"Deep Learning Approaches for Classifying Children With and Without Autism Spectrum Disorder Using Inertial Measurement Unit Hand Tracking Data: Comparative Study.","authors":"John Mutersbaugh, Wan-Chun Su, Anjana Bhat, Amir Gandjbakhche","doi":"10.2196/73440","DOIUrl":"10.2196/73440","url":null,"abstract":"<p><strong>Background: </strong>Autism spectrum disorder (ASD) is a prevalent neurodevelopmental condition that can be quite difficult to diagnose due to a lack of objective diagnostic methods in the currently used behavioral assessments. Recent work has shown that children with ASD have a higher incidence of motor control differences. A compilation of studies indicates that between 50% and 88% of the children with ASD have issues with movement control based on standardized motor assessments or parent-reported questionnaires.</p><p><strong>Objective: </strong>In this study, we assess a variety of deep learning approaches for the classification of ASD, utilizing data collected via inertial measurement unit (IMU) hand tracking during goal-directed arm movements.</p><p><strong>Methods: </strong>IMU hand tracking data were recorded from 41 school-aged children both with and without an ASD diagnosis to track their arm movements during a reach-to-clean up task. The IMU data were then preprocessed using a moving average and z score normalization to prepare the data for deep learning models. We evaluated the effectiveness of different deep learning models using the preprocessed data and a k-fold validation approach, as well as a patient-separated approach.</p><p><strong>Results: </strong>The best result was achieved with a convolutional autoencoder combined with long short-term memory layers, reaching an accuracy of 90.21% and an F1-score of 90.02%. Once the convolutional autoencoder+long short-term memory was determined to be the most effective model for this datatype, it was retrained and evaluated with a patient-separated dataset to assess the generalization capability of the model, achieving an accuracy of 91.87% and an F1-score of 93.66%.</p><p><strong>Conclusions: </strong>Our deep learning approach demonstrates that our models hold potential for facilitating ASD diagnosis in clinical settings. This work validates that there are significant differences between the physical movements of typically developing children and children with ASD, and these differences can be identified by analyzing hand-eye coordination skills. Additionally, we have validated that small-scale models can still achieve a high accuracy and good generalization when classifying medical data, opening the door for future research into diagnostic models that may not require massive amounts of data.</p>","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e73440"},"PeriodicalIF":3.8,"publicationDate":"2025-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12721220/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145806481","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
<p><strong>Background: </strong>The development of immunotherapy has provided new hope for patients with advanced gastric cancer (AGC). However, due to the high heterogeneity of the disease, the efficacy of first-line immunochemotherapy varies among patients. There is still a lack of simple and effective models to predict the efficacy of immunochemotherapy in this setting.</p><p><strong>Objective: </strong>This study aimed to identify critical factors and develop predictive models to evaluate the efficacy of first-line immunochemotherapy in patients with AGC using clinically available data. The goal was to offer evidence-based guidance for clinical practice and enable personalized treatment strategies.</p><p><strong>Methods: </strong>To evaluate the effectiveness of first-line immunochemotherapy in AGC, we retrospectively collected clinical data from The First Affiliated Hospital of Nanjing Medical University between January 2018 and October 2023. The data collected were divided into a training set (168/240, 70%) and an internal validation set (72/240, 30%). Additionally, a temporal validation cohort of 76 patients recruited from November 2023 to September 2024 was assembled to further evaluate the predictive performance of the models. We used univariate and multivariate Cox regression analyses, along with the least absolute shrinkage and selection operator (LASSO) regression, and integrated clinical expertise to identify key predictors of treatment efficacy and to construct the LASSO-Cox model. We developed 4 models (LASSO-Cox, random survival forest [RSF], extreme gradient boosting, and survival support vector machine) and evaluated their performance using the C-index, area under the curve (AUC), calibration curves, and decision curve analysis. The optimal model was interpreted using Shapley additive explanations, and its risk scores were used to stratify patients for Kaplan-Meier survival analysis.</p><p><strong>Results: </strong>Among the 4 prognostic models developed in this study, the RSF model demonstrated superior predictive accuracy and discrimination for progression-free survival, as evidenced by its higher AUC, concordance index, continuous AUC curves, and calibration curves compared with the other 3 models. Additionally, decision curve analysis showed that the RSF model offered greater net clinical benefit. The Shapley additive explanations results identified that age, histological subtype, the proportion of CD19<sup>+</sup> B cells, CD16<sup>+</sup>CD56<sup>+</sup> natural killer cells, and the presence of liver metastasis were key prognostic factors influencing patient outcomes. Patients in the low-risk group, as determined by the RSF model's risk score, exhibited a significantly higher progression-free survival rate than those in the high-risk group, further validating the value of the RSF model for risk stratification.</p><p><strong>Conclusions: </strong>This study is the first to use machine learning algorithms to develop a predi
背景:免疫疗法的发展为晚期胃癌患者提供了新的希望。然而,由于疾病的高度异质性,一线免疫化疗的疗效因患者而异。目前仍缺乏简单有效的模型来预测免疫化疗在这种情况下的疗效。目的:本研究旨在利用临床数据,确定关键因素并建立预测模型,以评估一线免疫化疗对AGC患者的疗效。目标是为临床实践提供循证指导,并实现个性化治疗策略。方法:为评价一线免疫化疗治疗AGC的有效性,回顾性收集南京医科大学第一附属医院2018年1月至2023年10月的临床资料。将收集到的数据分为训练集(168/ 240,70%)和内部验证集(72/ 240,30%)。此外,还收集了2023年11月至2024年9月期间招募的76名患者的时间验证队列,以进一步评估模型的预测性能。我们使用单变量和多变量Cox回归分析,以及最小绝对收缩和选择算子(LASSO)回归,并整合临床专业知识来确定治疗效果的关键预测因素,并构建LASSO-Cox模型。我们开发了LASSO-Cox、随机生存森林(RSF)、极端梯度增强(extreme gradient boosting)和生存支持向量机(survival support vector machine) 4个模型,并使用c指数、曲线下面积(AUC)、校准曲线和决策曲线分析来评估它们的性能。使用Shapley加性解释解释最优模型,并使用其风险评分对患者进行分层,进行Kaplan-Meier生存分析。结果:在本研究建立的4种预后模型中,RSF模型的AUC、一致性指数、连续AUC曲线和校准曲线均高于其他3种模型,对无进展生存期的预测准确性和判别性均优于其他3种模型。此外,决策曲线分析显示,RSF模型提供了更大的净临床效益。Shapley加性解释结果发现,年龄、组织学亚型、CD19+ B细胞、CD16+CD56+自然杀伤细胞的比例以及是否存在肝转移是影响患者预后的关键因素。根据RSF模型的风险评分,低危组患者的无进展生存率明显高于高危组,进一步验证了RSF模型在风险分层中的价值。结论:本研究首次使用机器学习算法建立了一线免疫化疗治疗AGC疗效的预测模型,并确定了治疗结果的关键预测因素。结果表明,RSF模型不仅可以对可能受益的患者进行精确分层,更重要的是,为个性化临床策略提供可量化的决策支持,强调了其在临床决策中的潜在价值。
{"title":"A Machine Learning Model Based on Clinical Factors to Predict the Efficacy of First-Line Immunochemotherapy for Patients With Advanced Gastric Cancer: Retrospective Study.","authors":"Xu Cheng, Ping Li, Enqing Meng, Xinyi Wu, Hao Wu","doi":"10.2196/82533","DOIUrl":"10.2196/82533","url":null,"abstract":"<p><strong>Background: </strong>The development of immunotherapy has provided new hope for patients with advanced gastric cancer (AGC). However, due to the high heterogeneity of the disease, the efficacy of first-line immunochemotherapy varies among patients. There is still a lack of simple and effective models to predict the efficacy of immunochemotherapy in this setting.</p><p><strong>Objective: </strong>This study aimed to identify critical factors and develop predictive models to evaluate the efficacy of first-line immunochemotherapy in patients with AGC using clinically available data. The goal was to offer evidence-based guidance for clinical practice and enable personalized treatment strategies.</p><p><strong>Methods: </strong>To evaluate the effectiveness of first-line immunochemotherapy in AGC, we retrospectively collected clinical data from The First Affiliated Hospital of Nanjing Medical University between January 2018 and October 2023. The data collected were divided into a training set (168/240, 70%) and an internal validation set (72/240, 30%). Additionally, a temporal validation cohort of 76 patients recruited from November 2023 to September 2024 was assembled to further evaluate the predictive performance of the models. We used univariate and multivariate Cox regression analyses, along with the least absolute shrinkage and selection operator (LASSO) regression, and integrated clinical expertise to identify key predictors of treatment efficacy and to construct the LASSO-Cox model. We developed 4 models (LASSO-Cox, random survival forest [RSF], extreme gradient boosting, and survival support vector machine) and evaluated their performance using the C-index, area under the curve (AUC), calibration curves, and decision curve analysis. The optimal model was interpreted using Shapley additive explanations, and its risk scores were used to stratify patients for Kaplan-Meier survival analysis.</p><p><strong>Results: </strong>Among the 4 prognostic models developed in this study, the RSF model demonstrated superior predictive accuracy and discrimination for progression-free survival, as evidenced by its higher AUC, concordance index, continuous AUC curves, and calibration curves compared with the other 3 models. Additionally, decision curve analysis showed that the RSF model offered greater net clinical benefit. The Shapley additive explanations results identified that age, histological subtype, the proportion of CD19<sup>+</sup> B cells, CD16<sup>+</sup>CD56<sup>+</sup> natural killer cells, and the presence of liver metastasis were key prognostic factors influencing patient outcomes. Patients in the low-risk group, as determined by the RSF model's risk score, exhibited a significantly higher progression-free survival rate than those in the high-risk group, further validating the value of the RSF model for risk stratification.</p><p><strong>Conclusions: </strong>This study is the first to use machine learning algorithms to develop a predi","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e82533"},"PeriodicalIF":3.8,"publicationDate":"2025-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12770927/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145812326","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
<p><strong>Background: </strong>The increasing use of real-time health data from wearable devices and self-reported questionnaires offers significant opportunities for preventive care in aging populations. However, current health data platforms often lack built-in mechanisms for data and model traceability, version control, and coordinated management of heterogeneous data streams, which are essential for clinical accountability, regulatory compliance, and reproducibility. The absence of these features limits the reuse of health data and the reproducibility of analytical workflows across research and clinical environments.</p><p><strong>Objective: </strong>This work presents DeltaTrace, a unified big data health platform designed with traceability as a key architectural feature. The platform integrates end-to-end tracking of data and model versions with real-time and batch processing capabilities. Built entirely on open source technologies, DeltaTrace combines components for data management, model management, orchestration, and visualization. The main objective is to demonstrate that embedding traceability within the architecture enables scalable, auditable, and version-controlled processing of health data, thereby facilitating reproducible analytics and long-term maintenance of health monitoring systems.</p><p><strong>Methods: </strong>DeltaTrace adopts a medallion architecture implemented with Delta Lake to ensure atomic and version-controlled data transformations. Apache Spark is used for distributed computation, Apache Kafka for continuous data ingestion, and Apache Airflow for orchestration of batch and streaming workflows. MLflow manages the lifecycle and versioning of machine learning models, while Grafana provides visualization dashboards for real-time and aggregated data inspection. The platform is evaluated using continuous physiological signals from wearable devices and batch-ingested questionnaire data, combining synthetic and real data from the LifeSnaps dataset. Performance tests are conducted on central processing unit-only servers with 8-core and 24-core configurations to assess ingestion, aggregation, visualization, and anomaly detection latency.</p><p><strong>Results: </strong>DeltaTrace supports continuous processing for approximately 1500 users with end-to-end delays below 10 minutes. Ingestion and visualization tasks operate between mean 4.9 (SD 0.12) and 7.5 (SD 0.28) minutes, while aggregation and anomaly detection required less than mean 5.6 (SD 0.04) and 10.5 (SD 1.70) minutes, respectively. Increasing from 8 to 24 cores improved ingestion and cleaning latency by up to 25% and anomaly detection performance by up to 50%. The system maintains consistent performance across different data types, processing modes, and loads.</p><p><strong>Conclusions: </strong>DeltaTrace provides a scalable and modular architecture that incorporates traceability as a core component together with functions for model management, orchestration, an
{"title":"Scalable Big Data Platform With End-to-End Traceability for Health Data Monitoring in Older Adults: Development and Performance Evaluation.","authors":"Ander Cejudo, Yone Tellechea, Amaia Calvo, Aitor Almeida, Cristina Martín, Andoni Beristain","doi":"10.2196/81701","DOIUrl":"10.2196/81701","url":null,"abstract":"<p><strong>Background: </strong>The increasing use of real-time health data from wearable devices and self-reported questionnaires offers significant opportunities for preventive care in aging populations. However, current health data platforms often lack built-in mechanisms for data and model traceability, version control, and coordinated management of heterogeneous data streams, which are essential for clinical accountability, regulatory compliance, and reproducibility. The absence of these features limits the reuse of health data and the reproducibility of analytical workflows across research and clinical environments.</p><p><strong>Objective: </strong>This work presents DeltaTrace, a unified big data health platform designed with traceability as a key architectural feature. The platform integrates end-to-end tracking of data and model versions with real-time and batch processing capabilities. Built entirely on open source technologies, DeltaTrace combines components for data management, model management, orchestration, and visualization. The main objective is to demonstrate that embedding traceability within the architecture enables scalable, auditable, and version-controlled processing of health data, thereby facilitating reproducible analytics and long-term maintenance of health monitoring systems.</p><p><strong>Methods: </strong>DeltaTrace adopts a medallion architecture implemented with Delta Lake to ensure atomic and version-controlled data transformations. Apache Spark is used for distributed computation, Apache Kafka for continuous data ingestion, and Apache Airflow for orchestration of batch and streaming workflows. MLflow manages the lifecycle and versioning of machine learning models, while Grafana provides visualization dashboards for real-time and aggregated data inspection. The platform is evaluated using continuous physiological signals from wearable devices and batch-ingested questionnaire data, combining synthetic and real data from the LifeSnaps dataset. Performance tests are conducted on central processing unit-only servers with 8-core and 24-core configurations to assess ingestion, aggregation, visualization, and anomaly detection latency.</p><p><strong>Results: </strong>DeltaTrace supports continuous processing for approximately 1500 users with end-to-end delays below 10 minutes. Ingestion and visualization tasks operate between mean 4.9 (SD 0.12) and 7.5 (SD 0.28) minutes, while aggregation and anomaly detection required less than mean 5.6 (SD 0.04) and 10.5 (SD 1.70) minutes, respectively. Increasing from 8 to 24 cores improved ingestion and cleaning latency by up to 25% and anomaly detection performance by up to 50%. The system maintains consistent performance across different data types, processing modes, and loads.</p><p><strong>Conclusions: </strong>DeltaTrace provides a scalable and modular architecture that incorporates traceability as a core component together with functions for model management, orchestration, an","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e81701"},"PeriodicalIF":3.8,"publicationDate":"2025-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12721222/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145806460","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sky Corby, Joan S Ash, Rebecca M Jungbauer, Gretchen Scholl, Sarah Florig, Vishnu Mohan, Jeffrey A Gold
Background: Electronic health records (EHRs) can aid in provider efficiency, but may also lead to unintended consequences, such as documentation burden and increased length of notes. To combat issues related to documentation, copying and pasting (CP) and copying or carrying forward (CF) are tools that have been used to aid in documentation burden. Multiple studies have identified the benefits and challenges of using these tools; however, few studies have identified the unintended consequences of CP and CF, and how the adoption of these tools may affect users.
Objective: The objective was to describe providers' perceptions and use of copying tools available in the EHR and describe their suggestions for improvement on these copying tools.
Methods: Research team members conducted semistructured interviews with faculty members, advanced practice providers, residents or fellow trainees, and medical students at a single academic health sciences center. The Diffusion of Innovations Theory of Unintended Consequences guided the analysis and interpretation of interview results.
Results: A total of 22 semistructured interviews were conducted in 2023 and analyzed during 2024. The findings showed that respondents use and value these tools for efficiency and communication purposes. The negative unintended consequences include inaccuracies and errors in documentation and increased patient safety risks. Some respondents experience inner angst or moral injury related to using CP/CF, but they feel that they must use them to satisfy organizational requirements surrounding documentation. The respondents suggested that artificial intelligence will likely help improve documentation tools, as would further training around these types of documentation tools.
Conclusions: Some respondents noted feeling both internal and external pressures that influenced when and how they use CP/CF. Respondents noted that they value EHR copying tools for efficiency purposes, but they also understand the risks involved. This tension may lead to moral angst or moral injury. They offered numerous suggestions for lowering the risk, especially by improving the documentation capabilities of the EHR through artificial intelligence. Future research should investigate both technical and educational solutions to relieve the documentation burden and moral angst they are experiencing.
{"title":"Copy Tools in the Electronic Health Record: Perceptions, Implications, and Future Directions.","authors":"Sky Corby, Joan S Ash, Rebecca M Jungbauer, Gretchen Scholl, Sarah Florig, Vishnu Mohan, Jeffrey A Gold","doi":"10.2196/78502","DOIUrl":"10.2196/78502","url":null,"abstract":"<p><strong>Background: </strong>Electronic health records (EHRs) can aid in provider efficiency, but may also lead to unintended consequences, such as documentation burden and increased length of notes. To combat issues related to documentation, copying and pasting (CP) and copying or carrying forward (CF) are tools that have been used to aid in documentation burden. Multiple studies have identified the benefits and challenges of using these tools; however, few studies have identified the unintended consequences of CP and CF, and how the adoption of these tools may affect users.</p><p><strong>Objective: </strong>The objective was to describe providers' perceptions and use of copying tools available in the EHR and describe their suggestions for improvement on these copying tools.</p><p><strong>Methods: </strong>Research team members conducted semistructured interviews with faculty members, advanced practice providers, residents or fellow trainees, and medical students at a single academic health sciences center. The Diffusion of Innovations Theory of Unintended Consequences guided the analysis and interpretation of interview results.</p><p><strong>Results: </strong>A total of 22 semistructured interviews were conducted in 2023 and analyzed during 2024. The findings showed that respondents use and value these tools for efficiency and communication purposes. The negative unintended consequences include inaccuracies and errors in documentation and increased patient safety risks. Some respondents experience inner angst or moral injury related to using CP/CF, but they feel that they must use them to satisfy organizational requirements surrounding documentation. The respondents suggested that artificial intelligence will likely help improve documentation tools, as would further training around these types of documentation tools.</p><p><strong>Conclusions: </strong>Some respondents noted feeling both internal and external pressures that influenced when and how they use CP/CF. Respondents noted that they value EHR copying tools for efficiency purposes, but they also understand the risks involved. This tension may lead to moral angst or moral injury. They offered numerous suggestions for lowering the risk, especially by improving the documentation capabilities of the EHR through artificial intelligence. Future research should investigate both technical and educational solutions to relieve the documentation burden and moral angst they are experiencing.</p>","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e78502"},"PeriodicalIF":3.8,"publicationDate":"2025-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12759298/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145795363","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Background: Accurately predicting the survival outcomes of patients with lung cancer receiving chemotherapy remains challenging.
Objective: To improve clinical management of this population, this study developed a multivariate machine learning (ML) model to assess all-cause mortality risk in chemotherapy-treated patients with lung cancer.
Methods: This study retrospectively recruited 1278 postchemotherapy patients with lung cancer from Guangzhou Chest Hospital between 2017 and 2019. Candidate features such as demographic characteristics, environmental exposures, clinical information, and patient-reported symptoms were collected via questionnaires and the electronic medical record system. The survival status and the deceased date were investigated twice a year. A total of 84 predictive models were constructed on the training set using 5 ML algorithms either individually or in pairwise combinations. The concordance index was used to identify the optimal model on the testing set, with performance validated via receiver operating characteristic curves, calibration curves, and decision curve analysis. Additionally, Shapley Additive Explanations and restricted cubic splines were applied for feature attribution analysis.
Results: The optimal model ultimately retained 21 prognosis-association features, including age, sex, BMI, smoking status, environmental smoke, the MD Anderson Symptom Inventory for Lung Cancer total score trajectories, cluster of differentiation 56, TNM stage, histology, and prechemotherapy blood biomarkers. On the testing set, the model acquired a concordance index of 0.702 (95% CI 0.652-0.753). The decision curves demonstrated positive clinical benefit when the risk thresholds were 0.40-0.69, 0.62-0.99, and 0.72-0.99 for 1-, 3-, and 5-year mortality predictions, respectively. The calibration curves showed that the predicted mortality probabilities fluctuated around the observed probabilities, and the Brier scores for 1-, 3-, and 5-year predictions were 0.20, 0.18, and 0.11, respectively. The area under the curve of the model was 0.740, 0.777, and 0.915 for 1-, 3-, and 5-year mortality predictions, respectively. Interpretability feature attribution analysis revealed that the significant features could predict all-cause mortality risk in chemotherapy-treated patients with lung cancer.
Conclusions: Our ML models exhibited acceptable discrimination, calibration, and clinical benefit in predicting the mortality risk of chemotherapy-treated patients with lung cancer, which could help clinicians in personalized prognostic management.
背景:准确预测接受化疗的肺癌患者的生存结果仍然具有挑战性。目的:为了改善这一人群的临床管理,本研究建立了一个多变量机器学习(ML)模型来评估化疗肺癌患者的全因死亡率风险。方法:本研究回顾性招募2017 - 2019年广州胸科医院肺癌化疗后患者1278例。候选特征,如人口统计学特征、环境暴露、临床信息和患者报告的症状,通过问卷调查和电子病历系统收集。每年调查两次患者的生存状况和死亡日期。在训练集上,使用5种ML算法单独或成对组合构建了84个预测模型。使用一致性指数在测试集上识别最优模型,并通过受试者工作特征曲线、校准曲线和决策曲线分析验证其性能。此外,应用Shapley加性解释和受限三次样条进行特征归因分析。结果:最佳模型最终保留了21个预后相关特征,包括年龄、性别、BMI、吸烟状况、环境烟雾、MD安德森肺癌症状量表总分轨迹、分化聚类56、TNM分期、组织学和化疗前血液生物标志物。在检验集上,模型的一致性指数为0.702 (95% CI 0.652-0.753)。当1年、3年和5年死亡率预测的风险阈值分别为0.40-0.69、0.62-0.99和0.72-0.99时,决策曲线显示出积极的临床获益。校正曲线显示,预测的死亡率概率在观测概率周围波动,1年、3年和5年预测的Brier评分分别为0.20、0.18和0.11。1年、3年和5年死亡率预测的曲线下面积分别为0.740、0.777和0.915。可解释性特征归因分析显示,显著性特征可以预测化疗肺癌患者的全因死亡风险。结论:我们的ML模型在预测化疗肺癌患者死亡风险方面表现出可接受的区分、校准和临床益处,可以帮助临床医生进行个性化预后管理。
{"title":"A Machine Learning Approach to Predicting Mortality Risk in Chemotherapy-Treated Lung Cancer: Machine Learning Model Development and Validation.","authors":"Jianjun Zou, Jinyi Huang, Katie Lu, Ao Lin, Chen Xie, Jinrong Zhang, Boqi Rao, Zhi Li, Dongming Xie, Ling Lu, Feng Luo, Jinbin Chen, Lei Yang, Fuman Qiu, Xin Zhang, Yibin Deng, Jiachun Lu","doi":"10.2196/72424","DOIUrl":"10.2196/72424","url":null,"abstract":"<p><strong>Background: </strong>Accurately predicting the survival outcomes of patients with lung cancer receiving chemotherapy remains challenging.</p><p><strong>Objective: </strong>To improve clinical management of this population, this study developed a multivariate machine learning (ML) model to assess all-cause mortality risk in chemotherapy-treated patients with lung cancer.</p><p><strong>Methods: </strong>This study retrospectively recruited 1278 postchemotherapy patients with lung cancer from Guangzhou Chest Hospital between 2017 and 2019. Candidate features such as demographic characteristics, environmental exposures, clinical information, and patient-reported symptoms were collected via questionnaires and the electronic medical record system. The survival status and the deceased date were investigated twice a year. A total of 84 predictive models were constructed on the training set using 5 ML algorithms either individually or in pairwise combinations. The concordance index was used to identify the optimal model on the testing set, with performance validated via receiver operating characteristic curves, calibration curves, and decision curve analysis. Additionally, Shapley Additive Explanations and restricted cubic splines were applied for feature attribution analysis.</p><p><strong>Results: </strong>The optimal model ultimately retained 21 prognosis-association features, including age, sex, BMI, smoking status, environmental smoke, the MD Anderson Symptom Inventory for Lung Cancer total score trajectories, cluster of differentiation 56, TNM stage, histology, and prechemotherapy blood biomarkers. On the testing set, the model acquired a concordance index of 0.702 (95% CI 0.652-0.753). The decision curves demonstrated positive clinical benefit when the risk thresholds were 0.40-0.69, 0.62-0.99, and 0.72-0.99 for 1-, 3-, and 5-year mortality predictions, respectively. The calibration curves showed that the predicted mortality probabilities fluctuated around the observed probabilities, and the Brier scores for 1-, 3-, and 5-year predictions were 0.20, 0.18, and 0.11, respectively. The area under the curve of the model was 0.740, 0.777, and 0.915 for 1-, 3-, and 5-year mortality predictions, respectively. Interpretability feature attribution analysis revealed that the significant features could predict all-cause mortality risk in chemotherapy-treated patients with lung cancer.</p><p><strong>Conclusions: </strong>Our ML models exhibited acceptable discrimination, calibration, and clinical benefit in predicting the mortality risk of chemotherapy-treated patients with lung cancer, which could help clinicians in personalized prognostic management.</p>","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e72424"},"PeriodicalIF":3.8,"publicationDate":"2025-12-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12757710/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145776708","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Background: Knee cartilage injury (KCI) poses significant challenges in the early clinical diagnosis process, primarily due to its high incidence, the complexity of healing, and the limited sensitivity of initial imaging modalities.
Objective: This study aims to employ magnetic resonance imaging and machine learning methods to enhance the classification accuracy of the classifier for KCI, improve the existing network structure, and demonstrate important clinical application value.
Methods: The proposed methodology is a multidimensional feature cross-level fusion classification network driven by the large separable kernel attention, which enables high-precision hierarchical diagnosis of KCI through deep learning. The network first fuses shallow high-resolution features with deep semantic features via the cross-level fusion module. Then, the large separable kernel attention module is embedded in the YOLOv8 network. This network utilizes the combined optimization of depth-separable and point-by-point convolutions to enhance features at multiple scales, thereby dramatically improving the hierarchical characterization of cartilage damage. Finally, five classifications of knee cartilage injuries are performed by classifiers.
Results: To overcome the limitations of network models trained with single-plane images, this study presents the first hospital-based multidimensional magnetic resonance imaging real dataset for KCI, on which the classification accuracy is 99.7%, the Kappa statistic is 99.6%, the F-measure is 99.7%, the sensitivity is 99.7%, and the specificity is 99.9%. The experimental results validate the feasibility of the proposed method.
Conclusions: The experimental outcomes confirm that the proposed methodology not only achieves exceptional performance in classifying knee cartilage injuries but also offers substantial improvements over existing techniques. This underscores its potential for clinical deployment in enhancing diagnostic precision and efficiency.
{"title":"Large Separable Kernel Attention-Driven Multidimensional Feature Cross-Level Fusion Classification Network of Knee Cartilage Injury: Algorithm Development and Validation.","authors":"Lirong Zhang, Hang Yu, Yating Yang","doi":"10.2196/79748","DOIUrl":"10.2196/79748","url":null,"abstract":"<p><strong>Background: </strong>Knee cartilage injury (KCI) poses significant challenges in the early clinical diagnosis process, primarily due to its high incidence, the complexity of healing, and the limited sensitivity of initial imaging modalities.</p><p><strong>Objective: </strong>This study aims to employ magnetic resonance imaging and machine learning methods to enhance the classification accuracy of the classifier for KCI, improve the existing network structure, and demonstrate important clinical application value.</p><p><strong>Methods: </strong>The proposed methodology is a multidimensional feature cross-level fusion classification network driven by the large separable kernel attention, which enables high-precision hierarchical diagnosis of KCI through deep learning. The network first fuses shallow high-resolution features with deep semantic features via the cross-level fusion module. Then, the large separable kernel attention module is embedded in the YOLOv8 network. This network utilizes the combined optimization of depth-separable and point-by-point convolutions to enhance features at multiple scales, thereby dramatically improving the hierarchical characterization of cartilage damage. Finally, five classifications of knee cartilage injuries are performed by classifiers.</p><p><strong>Results: </strong>To overcome the limitations of network models trained with single-plane images, this study presents the first hospital-based multidimensional magnetic resonance imaging real dataset for KCI, on which the classification accuracy is 99.7%, the Kappa statistic is 99.6%, the F-measure is 99.7%, the sensitivity is 99.7%, and the specificity is 99.9%. The experimental results validate the feasibility of the proposed method.</p><p><strong>Conclusions: </strong>The experimental outcomes confirm that the proposed methodology not only achieves exceptional performance in classifying knee cartilage injuries but also offers substantial improvements over existing techniques. This underscores its potential for clinical deployment in enhancing diagnostic precision and efficiency.</p>","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e79748"},"PeriodicalIF":3.8,"publicationDate":"2025-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12711134/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145776752","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Freddie Seba, Miriam Isola, Laura Mills, Mohan Zalake, Jacob Krive
Background: We designed learning assignments for students to develop knowledge, skills, and professional attitudes about generative artificial intelligence (AI) in 2 different Master's level courses in health informatics. Our innovative approach assumed that the students had no technical background or experience in using generative AI tools.
Objective: This study aims to offer generalizable methods and experiences on integration and assessment of generative AI content into the higher education's health informatics curricula. The study's central driver is the preparation of graduate students with generative AI tools, skills, ethical discernment, and critical thinking capacities aligned with the rapidly shifting job-market requirements, independent of graduate students' backgrounds and technical expertise.
Methods: During the semester, students completed a pretest and posttest to assess knowledge about generative AI. Reflections explored their expectations and experiences using generative AI to complete their assignments and projects during the semester. Strong emphasis was placed on building skills and professional attitudes by using generative AI. Student engagement in behavioral, emotional, and cognitive domains was explored via detailed analysis of student reflections by faculty.
Results: Students at the University of Illinois Chicago increased their knowledge about generative AI from 81% to 93% through research of the basic generative AI concepts, as evidenced from outcomes of the open-book pre-and posttests given at the beginning and end of the capstone course. University of San Francisco students also improved from 77% to 80% by the end of the semester. Faculty analysis of student reflections upon completion of the course revealed primary interests in the essentials of generative AI, AI transformations to information and knowledge, and organizational changes influenced by AI adoption in the health care organizations, with ethics being a primary driver of students' interests and engagement.
Conclusions: Data from student reflections provided insight into generative AI skills that students developed and that health informatics programs can consider incorporating into their curricula. Building competencies in generative AI will prepare students for the 21st century workforce and enable them to build skills employers are seeking in the new digital health environment.
背景:我们设计了学习作业,让学生在两个不同的健康信息学硕士课程中发展关于生成式人工智能(AI)的知识、技能和专业态度。我们的创新方法假设学生没有使用生成式人工智能工具的技术背景或经验。目的:本研究旨在为高等教育健康信息学课程整合与评估生成式人工智能内容提供可推广的方法和经验。该研究的核心驱动力是培养具有生成式人工智能工具、技能、道德洞察力和批判性思维能力的研究生,使其与快速变化的就业市场需求保持一致,独立于研究生的背景和技术专长。方法:在学期中,学生通过前测和后测来评估生成式人工智能的知识。反思探讨了他们在学期中使用生成式人工智能完成作业和项目的期望和经验。重点是通过使用生成式人工智能来培养技能和专业态度。通过教师对学生反映的详细分析,探讨了学生在行为、情感和认知领域的参与情况。结果:通过对生成式人工智能基本概念的研究,伊利诺伊大学芝加哥分校的学生对生成式人工智能的了解从81%增加到93%,这一点从顶点课程开始和结束时进行的开卷前测和后测的结果可以看出。到学期末,旧金山大学(University of San Francisco)的学生也从77%提高到了80%。教师对学生完成课程后反思的分析显示,他们对生成式人工智能的基本要素、人工智能向信息和知识的转化以及卫生保健组织中人工智能采用所影响的组织变革感兴趣,而道德是学生兴趣和参与的主要驱动力。结论:来自学生反思的数据提供了对学生发展的生成式人工智能技能的见解,健康信息学专业可以考虑将其纳入课程。在生成式人工智能方面培养能力将使学生为21世纪的劳动力做好准备,并使他们能够掌握雇主在新的数字健康环境中所寻求的技能。
{"title":"Incorporating Generative AI Into a Health Informatics Curriculum to Build 21st Century Competencies: Multisite Pre-Post Study.","authors":"Freddie Seba, Miriam Isola, Laura Mills, Mohan Zalake, Jacob Krive","doi":"10.2196/76507","DOIUrl":"10.2196/76507","url":null,"abstract":"<p><strong>Background: </strong>We designed learning assignments for students to develop knowledge, skills, and professional attitudes about generative artificial intelligence (AI) in 2 different Master's level courses in health informatics. Our innovative approach assumed that the students had no technical background or experience in using generative AI tools.</p><p><strong>Objective: </strong>This study aims to offer generalizable methods and experiences on integration and assessment of generative AI content into the higher education's health informatics curricula. The study's central driver is the preparation of graduate students with generative AI tools, skills, ethical discernment, and critical thinking capacities aligned with the rapidly shifting job-market requirements, independent of graduate students' backgrounds and technical expertise.</p><p><strong>Methods: </strong>During the semester, students completed a pretest and posttest to assess knowledge about generative AI. Reflections explored their expectations and experiences using generative AI to complete their assignments and projects during the semester. Strong emphasis was placed on building skills and professional attitudes by using generative AI. Student engagement in behavioral, emotional, and cognitive domains was explored via detailed analysis of student reflections by faculty.</p><p><strong>Results: </strong>Students at the University of Illinois Chicago increased their knowledge about generative AI from 81% to 93% through research of the basic generative AI concepts, as evidenced from outcomes of the open-book pre-and posttests given at the beginning and end of the capstone course. University of San Francisco students also improved from 77% to 80% by the end of the semester. Faculty analysis of student reflections upon completion of the course revealed primary interests in the essentials of generative AI, AI transformations to information and knowledge, and organizational changes influenced by AI adoption in the health care organizations, with ethics being a primary driver of students' interests and engagement.</p><p><strong>Conclusions: </strong>Data from student reflections provided insight into generative AI skills that students developed and that health informatics programs can consider incorporating into their curricula. Building competencies in generative AI will prepare students for the 21st century workforce and enable them to build skills employers are seeking in the new digital health environment.</p>","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e76507"},"PeriodicalIF":3.8,"publicationDate":"2025-12-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12707436/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145769877","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sizhe Jasmine Chen, Da Xu, Derek K Hu, Paul Jen-Hwa Hu, Ting-Shuo Huang
Background: Metabolic dysfunction-associated fatty liver disease (MAFLD) is a leading cause of chronic disease and can progress to liver fibrosis or hepatocellular carcinoma. Its subtypes-obese, diabetic, and lean-are associated with varying degrees of fibrotic burden and different complications, yet the existing analytics methods often overlook its multisystem nature, intraphenotype variability, and disease dynamics. These limitations hinder accurate risk stratification and restrict personalized intervention planning.
Objective: This study developed a novel, 2-stage, contrastive learning-based method to predict the phenotype of MAFLD among adults. This method leverages multiview contrastive learning; it models individual heterogeneities and important relationships in clinical and survey-based data to predict phenotypes among adults, thus supporting clinical decision-making and personalized care.
Methods: Demographic, clinical, lifestyle, and genetic family history data of 4408 adults revealed how capturing essential relationships in patient data from different sources can transform individual-level representations into multiple, complementary views. Evaluation of the predictive efficacy of the proposed method in comparison with 8 prevalent methods relied on recall, precision, F1-score, and area under the curve values. Moreover, a Shapley additive explanation analysis was performed for interpretability.
Results: The proposed method consistently and significantly outperformed all benchmark methods. It attained the highest F1-score, showing a 32.8% improvement for nondiabetic MAFLD (0.531 vs 0.400) and 30.4% improvement for diabetic MAFLD (0.519 vs 0.398) over the respective best-performing benchmark. The results underscore the clinical value and utility of integrating clinical and survey-based data in the prediction of MAFLD phenotypes among adults.
Conclusions: The proposed method is a viable approach for MAFLD phenotype prediction. It is more effective in identifying at-risk adults than many prevalent data-driven analytics methods and thereby can enhance clinical decision-making and support patient-centric care and management.
背景:代谢功能障碍相关脂肪性肝病(MAFLD)是慢性疾病的主要原因,可发展为肝纤维化或肝细胞癌。其亚型(肥胖、糖尿病和消瘦)与不同程度的纤维化负担和不同的并发症相关,但现有的分析方法往往忽略了其多系统性质、表型内变异性和疾病动力学。这些限制阻碍了准确的风险分层,并限制了个性化的干预计划。目的:本研究开发了一种新的、两阶段的、基于对比学习的方法来预测成人MAFLD的表型。这种方法利用了多视角对比学习;它模拟了临床和基于调查的数据中的个体异质性和重要关系,以预测成人的表型,从而支持临床决策和个性化护理。方法:4408名成年人的人口统计、临床、生活方式和遗传家族史数据揭示了如何从不同来源的患者数据中捕捉基本关系,将个人层面的表征转化为多个互补的观点。与8种流行方法相比,该方法的预测效果评估依赖于召回率、精度、f1评分和曲线下面积值。此外,对可解释性进行了Shapley加性解释分析。结果:所提方法一致且显著优于所有基准方法。它获得了最高的f1评分,与各自的最佳表现基准相比,非糖尿病性MAFLD改善了32.8% (0.531 vs 0.400),糖尿病性MAFLD改善了30.4% (0.519 vs 0.398)。研究结果强调了综合临床和基于调查的数据预测成人MAFLD表型的临床价值和效用。结论:该方法是一种预测MAFLD表型的可行方法。它比许多流行的数据驱动分析方法更有效地识别有风险的成年人,从而可以增强临床决策并支持以患者为中心的护理和管理。
{"title":"Predicting Metabolic Dysfunction-Associated Fatty Liver Disease Phenotypes Among Adults: 2-Stage Contrastive Learning Method.","authors":"Sizhe Jasmine Chen, Da Xu, Derek K Hu, Paul Jen-Hwa Hu, Ting-Shuo Huang","doi":"10.2196/75747","DOIUrl":"10.2196/75747","url":null,"abstract":"<p><strong>Background: </strong>Metabolic dysfunction-associated fatty liver disease (MAFLD) is a leading cause of chronic disease and can progress to liver fibrosis or hepatocellular carcinoma. Its subtypes-obese, diabetic, and lean-are associated with varying degrees of fibrotic burden and different complications, yet the existing analytics methods often overlook its multisystem nature, intraphenotype variability, and disease dynamics. These limitations hinder accurate risk stratification and restrict personalized intervention planning.</p><p><strong>Objective: </strong>This study developed a novel, 2-stage, contrastive learning-based method to predict the phenotype of MAFLD among adults. This method leverages multiview contrastive learning; it models individual heterogeneities and important relationships in clinical and survey-based data to predict phenotypes among adults, thus supporting clinical decision-making and personalized care.</p><p><strong>Methods: </strong>Demographic, clinical, lifestyle, and genetic family history data of 4408 adults revealed how capturing essential relationships in patient data from different sources can transform individual-level representations into multiple, complementary views. Evaluation of the predictive efficacy of the proposed method in comparison with 8 prevalent methods relied on recall, precision, F1-score, and area under the curve values. Moreover, a Shapley additive explanation analysis was performed for interpretability.</p><p><strong>Results: </strong>The proposed method consistently and significantly outperformed all benchmark methods. It attained the highest F1-score, showing a 32.8% improvement for nondiabetic MAFLD (0.531 vs 0.400) and 30.4% improvement for diabetic MAFLD (0.519 vs 0.398) over the respective best-performing benchmark. The results underscore the clinical value and utility of integrating clinical and survey-based data in the prediction of MAFLD phenotypes among adults.</p><p><strong>Conclusions: </strong>The proposed method is a viable approach for MAFLD phenotype prediction. It is more effective in identifying at-risk adults than many prevalent data-driven analytics methods and thereby can enhance clinical decision-making and support patient-centric care and management.</p>","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e75747"},"PeriodicalIF":3.8,"publicationDate":"2025-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12702840/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145758554","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Riley Bove, Luca Capezzuto, Imogen West, Simon Dryden, Saira Ghafur, Jack Halligan, Stanislas Hubeaux, Agne Kazlauskaite
<p><strong>Background: </strong>Efforts are being made to integrate digital health technologies into clinical care for multiple sclerosis (MS) to improve patient monitoring. Efficiently probing how they might impact clinical care could streamline digital tool development. The Floodlight digital tool, comprising 5 smartphone sensor-based tests, was used to generate health-related data on patient function and symptoms in a clinical simulation.</p><p><strong>Objective: </strong>The study had 3 objectives: (1) assess the utility of simulated clinical encounters as a research methodology for exploring the introduction of digital health technologies into clinical practice in MS, (2) confirm the fidelity of the simulated environment and patient cases developed and understand what metrics (eg, workflow, comprehensive evaluation) could be generated, and (3) generate insights into the utility of digitally collected data, including usability, clinical decision contribution, and impact on workflows, in clinical practice.</p><p><strong>Methods: </strong>A total of 2 patient cases consisting of clinical, radiological, and digital health data were developed with clinician input. US-based neurologists prepared for and conducted 2 simulated teleconsultations each, with an actor briefed on case profiles. Floodlight data were available, via the Floodlight MS™ Health Care Professional Portal, for 1 of the 2 consultations. Participant neurologists completed interviews and surveys assessing the fidelity of the cases presented, user experience and workflow metrics, patient concerns identified, care decisions made, and confidence in making decisions.</p><p><strong>Results: </strong>All 10 neurologists indicated that the simulations were high-fidelity representations of real consultations. Using the Floodlight technology for the first time, median time taken to prepare for and conduct the consultation was ~1.7-2 minutes longer, with slightly greater mental effort reported by participants, compared with not using the tool. The Floodlight MS Health Care Professional Portal scored an "above average" 79 on the System Usability Scale and an "acceptable" Net Promoter Score of 10. In total, 6 of the 10 neurologists "strongly agreed" that it was easier and quicker to identify patient concerns when they had access to the patient-generated Floodlight data to prepare for their encounters than when they did not. Overall, more care and management decisions were taken when the digital tool was used (37 vs 29). Of those 37 decisions, Floodlight data were reported as a trigger for 20 decisions, always in combination with other elements including patient history (20/20) and clinical exam findings (9/20).</p><p><strong>Conclusions: </strong>These findings advance our understanding of clinical simulation as a method for evaluating digital tools and other innovative technologies for MS care. High-fidelity patient cases could be provided for the mock teleconsultations, and the simulated clin
背景:人们正在努力将数字健康技术整合到多发性硬化症(MS)的临床护理中,以改善患者的监测。有效地探索它们如何影响临床护理可以简化数字工具的开发。泛光灯数字工具包括5个基于智能手机传感器的测试,用于在临床模拟中生成有关患者功能和症状的健康相关数据。目的:本研究有3个目的:(1)评估模拟临床接触的效用,作为探索将数字健康技术引入MS临床实践的研究方法;(2)确认模拟环境和患者病例的保真度,并了解可以产生哪些指标(例如,工作流,综合评估);(3)对数字收集数据的效用产生见解,包括可用性、临床决策贡献和对工作流程的影响。临床实践中。方法:在临床医生的输入下,编制2例患者的临床、放射学和数字健康资料。美国的神经学家准备并进行了两次模拟远程会诊,并向一名演员简要介绍了病例概况。2次咨询中的1次可通过Floodlight MS™医疗保健专业门户获得泛光灯数据。参与的神经科医生完成了访谈和调查,评估了所呈现病例的保真度、用户体验和工作流程指标、确定的患者关注点、做出的护理决策以及做出决策的信心。结果:所有10名神经科医生表示,模拟是真实咨询的高保真表现。第一次使用泛光灯技术,准备和进行咨询的平均时间约为1.7-2分钟,与不使用该工具相比,参与者报告的心理努力略大。泛光灯MS医疗保健专业门户网站在系统可用性量表上获得了79分“高于平均水平”,净推荐值为10分,“可接受”。总的来说,10位神经科医生中有6位“强烈同意”,当他们可以访问患者生成的泛光灯数据以为他们的遭遇做准备时,比没有访问时更容易、更快地识别患者的担忧。总体而言,当使用数字工具时,采取了更多的谨慎和管理决策(37 vs 29)。在这37项决策中,泛光灯数据被报告为20项决策的触发因素,这些决策总是与其他因素相结合,包括患者病史(20/20)和临床检查结果(9/20)。结论:这些发现促进了我们对临床模拟作为评估MS护理的数字工具和其他创新技术的方法的理解。可以为模拟远程会诊提供高保真的患者病例,模拟临床环境有助于评估新的数字工具的可用性和实用性,从而为神经科医生如何访问和利用数字数据来支持常规MS护理提供初步证据。
{"title":"Simulation of Clinical Visits as a Novel Approach to Evaluate Digital Health in Multiple Sclerosis: Simulation Study.","authors":"Riley Bove, Luca Capezzuto, Imogen West, Simon Dryden, Saira Ghafur, Jack Halligan, Stanislas Hubeaux, Agne Kazlauskaite","doi":"10.2196/67845","DOIUrl":"10.2196/67845","url":null,"abstract":"<p><strong>Background: </strong>Efforts are being made to integrate digital health technologies into clinical care for multiple sclerosis (MS) to improve patient monitoring. Efficiently probing how they might impact clinical care could streamline digital tool development. The Floodlight digital tool, comprising 5 smartphone sensor-based tests, was used to generate health-related data on patient function and symptoms in a clinical simulation.</p><p><strong>Objective: </strong>The study had 3 objectives: (1) assess the utility of simulated clinical encounters as a research methodology for exploring the introduction of digital health technologies into clinical practice in MS, (2) confirm the fidelity of the simulated environment and patient cases developed and understand what metrics (eg, workflow, comprehensive evaluation) could be generated, and (3) generate insights into the utility of digitally collected data, including usability, clinical decision contribution, and impact on workflows, in clinical practice.</p><p><strong>Methods: </strong>A total of 2 patient cases consisting of clinical, radiological, and digital health data were developed with clinician input. US-based neurologists prepared for and conducted 2 simulated teleconsultations each, with an actor briefed on case profiles. Floodlight data were available, via the Floodlight MS™ Health Care Professional Portal, for 1 of the 2 consultations. Participant neurologists completed interviews and surveys assessing the fidelity of the cases presented, user experience and workflow metrics, patient concerns identified, care decisions made, and confidence in making decisions.</p><p><strong>Results: </strong>All 10 neurologists indicated that the simulations were high-fidelity representations of real consultations. Using the Floodlight technology for the first time, median time taken to prepare for and conduct the consultation was ~1.7-2 minutes longer, with slightly greater mental effort reported by participants, compared with not using the tool. The Floodlight MS Health Care Professional Portal scored an \"above average\" 79 on the System Usability Scale and an \"acceptable\" Net Promoter Score of 10. In total, 6 of the 10 neurologists \"strongly agreed\" that it was easier and quicker to identify patient concerns when they had access to the patient-generated Floodlight data to prepare for their encounters than when they did not. Overall, more care and management decisions were taken when the digital tool was used (37 vs 29). Of those 37 decisions, Floodlight data were reported as a trigger for 20 decisions, always in combination with other elements including patient history (20/20) and clinical exam findings (9/20).</p><p><strong>Conclusions: </strong>These findings advance our understanding of clinical simulation as a method for evaluating digital tools and other innovative technologies for MS care. High-fidelity patient cases could be provided for the mock teleconsultations, and the simulated clin","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e67845"},"PeriodicalIF":3.8,"publicationDate":"2025-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12697915/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145745868","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ya-Han Hu, Yi-Ying Cheng, Chung-Ching Lan, Yu-Hsiang Su, Sheng-Feng Sung
<p><strong>Background: </strong>Clinical trial eligibility screening using electronic medical records (EMRs) is challenging due to the complexity of patient data and the varied clinical terminologies. Manual screening is time-consuming, requires specialized knowledge, and can lead to inconsistent participant selection, potentially compromising patient safety and research outcomes. This is critical in time-sensitive conditions like acute ischemic stroke. While computerized clinical decision support tools offer solutions, most require software engineering expertise to update, limiting their practical utility when eligibility criteria change.</p><p><strong>Objective: </strong>We developed and evaluated the intelligent trial eligibility screening tool (iTEST), which combines natural language processing with a block-based visual programming interface designed to enable clinicians to create and modify eligibility screening rules independently. In this study, we assessed iTEST's rule evaluation module using pre-configured rules and compared its effectiveness with that of standard EMR interfaces.</p><p><strong>Methods: </strong>We conducted an experiment at a tertiary teaching hospital in Taiwan with 12 clinicians using a 2-period crossover design. The clinicians assessed the eligibility of 4 patients with stroke for 2 clinical trials using both standard EMR and iTEST in a counterbalanced order, resulting in 48 evaluation scenarios. The iTEST comprised a rule authoring module using Google Blockly and a rule evaluation module utilizing MetaMap Lite for extracting medical concepts from unstructured EMR documents and structured laboratory data. Primary outcomes included accuracy in determining eligibility. Secondary outcomes measured task completion time, cognitive workload using the National Aeronautics and Space Administration Task Load Index scale (range 0-100, with lower scores indicating a lower cognitive workload), and system usability through the system usability scale (range: 0-100, with higher scores indicating higher system usability).</p><p><strong>Results: </strong>The iTEST significantly improved accuracy scores (from 0.91 to 1.00, P<.001) and reduced completion time (from 3.18 to 2.44 min, P=.004) compared to the standard EMR interface. Users reported lower cognitive workload (National Aeronautics and Space Administration Task Load Index scale, 39.7 vs 62.8, P=.02) and higher system usability scale scores (71.3 vs 46.3, P=.01) with the iTEST. Particularly notable improvements in perceived cognitive workload were observed in temporal demand, effort, and frustration levels.</p><p><strong>Conclusions: </strong>The iTEST demonstrated superior performance in clinical trial eligibility screening, delivering improved accuracy, reduced completion time, lower cognitive workload, and better usability when evaluating preconfigured eligibility rules. The improved accuracy is critical for patient safety, as the misidentification of eligibility criteria cou
{"title":"An Intelligent Trial Eligibility Screening Tool Using Natural Language Processing With a Block-Based Visual Programming Interface: Development and Usability Study.","authors":"Ya-Han Hu, Yi-Ying Cheng, Chung-Ching Lan, Yu-Hsiang Su, Sheng-Feng Sung","doi":"10.2196/80072","DOIUrl":"10.2196/80072","url":null,"abstract":"<p><strong>Background: </strong>Clinical trial eligibility screening using electronic medical records (EMRs) is challenging due to the complexity of patient data and the varied clinical terminologies. Manual screening is time-consuming, requires specialized knowledge, and can lead to inconsistent participant selection, potentially compromising patient safety and research outcomes. This is critical in time-sensitive conditions like acute ischemic stroke. While computerized clinical decision support tools offer solutions, most require software engineering expertise to update, limiting their practical utility when eligibility criteria change.</p><p><strong>Objective: </strong>We developed and evaluated the intelligent trial eligibility screening tool (iTEST), which combines natural language processing with a block-based visual programming interface designed to enable clinicians to create and modify eligibility screening rules independently. In this study, we assessed iTEST's rule evaluation module using pre-configured rules and compared its effectiveness with that of standard EMR interfaces.</p><p><strong>Methods: </strong>We conducted an experiment at a tertiary teaching hospital in Taiwan with 12 clinicians using a 2-period crossover design. The clinicians assessed the eligibility of 4 patients with stroke for 2 clinical trials using both standard EMR and iTEST in a counterbalanced order, resulting in 48 evaluation scenarios. The iTEST comprised a rule authoring module using Google Blockly and a rule evaluation module utilizing MetaMap Lite for extracting medical concepts from unstructured EMR documents and structured laboratory data. Primary outcomes included accuracy in determining eligibility. Secondary outcomes measured task completion time, cognitive workload using the National Aeronautics and Space Administration Task Load Index scale (range 0-100, with lower scores indicating a lower cognitive workload), and system usability through the system usability scale (range: 0-100, with higher scores indicating higher system usability).</p><p><strong>Results: </strong>The iTEST significantly improved accuracy scores (from 0.91 to 1.00, P<.001) and reduced completion time (from 3.18 to 2.44 min, P=.004) compared to the standard EMR interface. Users reported lower cognitive workload (National Aeronautics and Space Administration Task Load Index scale, 39.7 vs 62.8, P=.02) and higher system usability scale scores (71.3 vs 46.3, P=.01) with the iTEST. Particularly notable improvements in perceived cognitive workload were observed in temporal demand, effort, and frustration levels.</p><p><strong>Conclusions: </strong>The iTEST demonstrated superior performance in clinical trial eligibility screening, delivering improved accuracy, reduced completion time, lower cognitive workload, and better usability when evaluating preconfigured eligibility rules. The improved accuracy is critical for patient safety, as the misidentification of eligibility criteria cou","PeriodicalId":56334,"journal":{"name":"JMIR Medical Informatics","volume":"13 ","pages":"e80072"},"PeriodicalIF":3.8,"publicationDate":"2025-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12698033/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145745925","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}