Pub Date : 2025-03-18DOI: 10.1016/j.mcpdig.2025.100216
Mirjam M. Jern-Matintupa MD, MPH , Anita M. Riipinen MD, PhD , Merja K. Laine MD, PhD
Objective
To assess the existing body of evidence and impact of digital interventions on occupational health care.
Methods
The search strategy and review process were conducted in accordance with the PRISMA guidelines. The search was carried out during a period from January 1, 2013 to June 5, 2023, using the SCOPUS and Ovid Medline databases. After the identification of the relevant records, screening was conducted in 3 stages, following specific predetermined inclusion and exclusion criteria. A data-extraction model was created on the basis of the aim of the review. The quality of the selected studies was evaluated using the Effective Public Health Practice framework. Owing to the heterogeneity of the outcome measures, we used narrative synthesis to summarize the findings.
Results
We identified 382 records in SCOPUS and 441 in Ovid Medline. We selected 54 studies to be included in the evidence synthesis. The health targets of the interventions varied widely, but we identified 2 main focus areas: sedentary behavior (n=17, 32%) and mental health (n=14, 26%). Even when the studies had the same health target, the outcomes and chosen measures varied widely. Given the considerable effect of the primary outcome, mental health appears to be a good target for digital interventions. Online training and computer software could be especially effective.
Conclusion
The potential positive impact of digital interventions on mental health, especially online training, should be leveraged by health care professionals and providers. In order to provide more specific recommendations for health care professionals, occupational health care researchers should strive for consensus on outcome measures.
{"title":"Impact of Digital Interventions in Occupational Health Care: A Systematic Review","authors":"Mirjam M. Jern-Matintupa MD, MPH , Anita M. Riipinen MD, PhD , Merja K. Laine MD, PhD","doi":"10.1016/j.mcpdig.2025.100216","DOIUrl":"10.1016/j.mcpdig.2025.100216","url":null,"abstract":"<div><h3>Objective</h3><div>To assess the existing body of evidence and impact of digital interventions on occupational health care.</div></div><div><h3>Methods</h3><div>The search strategy and review process were conducted in accordance with the PRISMA guidelines. The search was carried out during a period from January 1, 2013 to June 5, 2023, using the SCOPUS and Ovid Medline databases. After the identification of the relevant records, screening was conducted in 3 stages, following specific predetermined inclusion and exclusion criteria. A data-extraction model was created on the basis of the aim of the review. The quality of the selected studies was evaluated using the Effective Public Health Practice framework. Owing to the heterogeneity of the outcome measures, we used narrative synthesis to summarize the findings.</div></div><div><h3>Results</h3><div>We identified 382 records in SCOPUS and 441 in Ovid Medline. We selected 54 studies to be included in the evidence synthesis. The health targets of the interventions varied widely, but we identified 2 main focus areas: sedentary behavior (n=17, 32%) and mental health (n=14, 26%). Even when the studies had the same health target, the outcomes and chosen measures varied widely. Given the considerable effect of the primary outcome, mental health appears to be a good target for digital interventions. Online training and computer software could be especially effective.</div></div><div><h3>Conclusion</h3><div>The potential positive impact of digital interventions on mental health, especially online training, should be leveraged by health care professionals and providers. In order to provide more specific recommendations for health care professionals, occupational health care researchers should strive for consensus on outcome measures.</div></div>","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"3 2","pages":"Article 100216"},"PeriodicalIF":0.0,"publicationDate":"2025-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143792485","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-03-12DOI: 10.1016/j.mcpdig.2025.100215
{"title":"Erratum to Leveraging the Metaverse for Enhanced Longevity as a Component of Health 4.0 [Mayo Clinic Proceedings: Digital Health. 2024;2:139-151]","authors":"","doi":"10.1016/j.mcpdig.2025.100215","DOIUrl":"10.1016/j.mcpdig.2025.100215","url":null,"abstract":"","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"3 2","pages":"Article 100215"},"PeriodicalIF":0.0,"publicationDate":"2025-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143705167","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-03-12DOI: 10.1016/j.mcpdig.2025.100202
Jessica K. Lu MEng , Weilan Wang PhD , Jorming Goh PhD , Andrea B. Maier MD, PhD
Continuous monitoring of cardiovascular functions can provide crucial insights into the health status and lifestyle behaviors of an individual. Wearable devices offer a convenient and cost-effective solution for collecting cardiovascular measurements outside clinical settings. However, the abundance of available devices poses challenges for researchers, health care professionals, and device users in selecting the most suitable one. This article illustrates the application of a practical guide for selecting wearable devices for the continuous monitoring of cardiovascular functions in community-dwelling adults who are generally healthy or have minimal, well-managed chronic conditions. An initial systematic review of the literature revealed 216 devices, each of which were assessed on the basis of 5 core criteria from the guide: (1) continuous monitoring capability, (2) device availability and suitability, (3) technical performance (accuracy and precision), (4) feasibility of use, and (5) cost evaluation. From the 216 devices, there were 136 devices capable of continuous monitoring. After the exclusion of unavailable and unsuitable devices, 53 devices underwent validation assessment of accuracy and precision. Although COSMIN criteria were applied to evaluate technical performance, a lack of validation for certain devices limits a comprehensive evaluation. After selection of valid devices, the feasibility and cost of 20 devices were examined. Wearable devices, such as the Apple Watch Series 9, Fitbit Charge 6, Garmin vívosmart 5, and Oura Ring Gen3, emerged as suitable devices to measure cardiovascular function in community-dwelling adults. The systematic process for device selection could also be applied to select wearable devices for the measurement of other physiologic variables and lifestyle behaviors.
持续监测心血管功能可以对个人的健康状况和生活方式行为提供重要的见解。可穿戴设备为在临床环境之外收集心血管测量数据提供了一种方便且经济高效的解决方案。然而，大量的可用设备给研究人员、卫生保健专业人员和设备用户在选择最合适的设备方面带来了挑战。本文阐述了一种实用指南的应用，用于选择可穿戴设备，用于持续监测社区居住成年人的心血管功能，这些成年人通常健康或有最小的、管理良好的慢性病。对文献的初步系统回顾显示了216个设备，每个设备都是根据指南中的5个核心标准进行评估的：(1)持续监测能力，(2)设备可用性和适用性，(3)技术性能（准确性和精密度），(4)使用可行性，(5)成本评估。在216个装置中，有136个装置能够持续监测。在排除不可用和不合适的器械后，53个器械进行了准确性和精密度的验证评估。虽然采用了COSMIN标准来评价技术性能，但缺乏对某些设备的验证限制了全面评价。在选择了有效装置后，对20种装置的可行性和成本进行了考察。可穿戴设备，如Apple Watch Series 9、Fitbit Charge 6、Garmin vívosmart 5和Oura Ring Gen3，成为测量社区居民心血管功能的合适设备。设备选择的系统过程也可以应用于选择可穿戴设备来测量其他生理变量和生活方式行为。
{"title":"Selecting Wearable Devices to Measure Cardiovascular Functions in Community-Dwelling Adults: Application of a Practical Guide for Device Selection","authors":"Jessica K. Lu MEng , Weilan Wang PhD , Jorming Goh PhD , Andrea B. Maier MD, PhD","doi":"10.1016/j.mcpdig.2025.100202","DOIUrl":"10.1016/j.mcpdig.2025.100202","url":null,"abstract":"<div><div>Continuous monitoring of cardiovascular functions can provide crucial insights into the health status and lifestyle behaviors of an individual. Wearable devices offer a convenient and cost-effective solution for collecting cardiovascular measurements outside clinical settings. However, the abundance of available devices poses challenges for researchers, health care professionals, and device users in selecting the most suitable one. This article illustrates the application of a practical guide for selecting wearable devices for the continuous monitoring of cardiovascular functions in community-dwelling adults who are generally healthy or have minimal, well-managed chronic conditions. An initial systematic review of the literature revealed 216 devices, each of which were assessed on the basis of 5 core criteria from the guide: (1) continuous monitoring capability, (2) device availability and suitability, (3) technical performance (accuracy and precision), (4) feasibility of use, and (5) cost evaluation. From the 216 devices, there were 136 devices capable of continuous monitoring. After the exclusion of unavailable and unsuitable devices, 53 devices underwent validation assessment of accuracy and precision. Although COSMIN criteria were applied to evaluate technical performance, a lack of validation for certain devices limits a comprehensive evaluation. After selection of valid devices, the feasibility and cost of 20 devices were examined. Wearable devices, such as the Apple Watch Series 9, Fitbit Charge 6, Garmin vívosmart 5, and Oura Ring Gen3, emerged as suitable devices to measure cardiovascular function in community-dwelling adults. The systematic process for device selection could also be applied to select wearable devices for the measurement of other physiologic variables and lifestyle behaviors.</div></div>","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"3 2","pages":"Article 100202"},"PeriodicalIF":0.0,"publicationDate":"2025-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143697823","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-03-08DOI: 10.1016/j.mcpdig.2025.100206
Canio Martinelli MD , Antonio Giordano MD , Vincenzo Carnevale PhD , Sharon Raffaella Burk PhD , Lavinia Porto MD , Giuseppe Vizzielli MD , Alfredo Ercoli MD
Objective
To systematically evaluate the performance of artificial intelligence (AI) large language models (LLMs) compared with obstetrics-gynecology residents in clinical decision-making, examining diagnostic accuracy and error patterns across linguistic domains, time constraints, and experience levels.
Patients and Methods
In this cross-sectional study, we evaluated 8 AI LLMs and 24 obstetrics-gynecology residents (Years 1-5) using 60 standardized clinical scenarios. Most AI LLMs and all residents were assessed in May 2024, whereas chat GPT-01-preview, chat-GPT4o, and Claude Sonnet 3.5 were evaluated in November 2024. The assessment framework incorporated English and Italian scenarios under both timed and untimed conditions, along with systematic error pattern analysis. The primary outcome was diagnostic accuracy; secondary end points included AI system stratification, resident progression, language impact, time pressure effects, and integration potential.
Results
The AI LLMs reported superior overall accuracy (73.75%; 95% confidence interval [CI], 69.64%-77.49%) compared with residents (65.35%; 95% CI, 62.85%-67.76%; P<.001). High-performing AI systems (ChatGPT-01-preview, GPT4o, and Claude Sonnet 3.5) achieved consistently high cross-linguistic accuracy (88.33%) with minimal language impact (6.67%±0.00%). Resident performance declined significantly under time constraints (from 73.2% to 56.5% adjusted accuracy; Cohen’s d=1.009; P<.001), whereas AI systems reported lesser deterioration. Error pattern analysis indicated a moderate correlation between AI and human reasoning (r=0.666; P<.001). Residents exhibited systematic progression from year 1 (44.7%) to year 5 (87.1%). Integration analysis found variable benefits across training levels, with maximum enhancement in early-career residents (+29.7%; P<.001).
Conclusion
High-performing AI LLMs reported strong diagnostic accuracy and resilience under linguistic and temporal pressures. These findings suggest that AI-enhanced decision-making may offer particular benefits in obstetrics and gynecology training programs, especially for junior residents, by improving diagnostic consistency and potentially reducing cognitive load in time-sensitive clinical settings.
{"title":"The PERFORM Study: Artificial Intelligence Versus Human Residents in Cross-Sectional Obstetrics-Gynecology Scenarios Across Languages and Time Constraints","authors":"Canio Martinelli MD , Antonio Giordano MD , Vincenzo Carnevale PhD , Sharon Raffaella Burk PhD , Lavinia Porto MD , Giuseppe Vizzielli MD , Alfredo Ercoli MD","doi":"10.1016/j.mcpdig.2025.100206","DOIUrl":"10.1016/j.mcpdig.2025.100206","url":null,"abstract":"<div><h3>Objective</h3><div>To systematically evaluate the performance of artificial intelligence (AI) large language models (LLMs) compared with obstetrics-gynecology residents in clinical decision-making, examining diagnostic accuracy and error patterns across linguistic domains, time constraints, and experience levels.</div></div><div><h3>Patients and Methods</h3><div>In this cross-sectional study, we evaluated 8 AI LLMs and 24 obstetrics-gynecology residents (Years 1-5) using 60 standardized clinical scenarios. Most AI LLMs and all residents were assessed in May 2024, whereas chat GPT-01-preview, chat-GPT4o, and Claude Sonnet 3.5 were evaluated in November 2024. The assessment framework incorporated English and Italian scenarios under both timed and untimed conditions, along with systematic error pattern analysis. The primary outcome was diagnostic accuracy; secondary end points included AI system stratification, resident progression, language impact, time pressure effects, and integration potential.</div></div><div><h3>Results</h3><div>The AI LLMs reported superior overall accuracy (73.75%; 95% confidence interval [CI], 69.64%-77.49%) compared with residents (65.35%; 95% CI, 62.85%-67.76%; <em>P</em><.001). High-performing AI systems (ChatGPT-01-preview, GPT4o, and Claude Sonnet 3.5) achieved consistently high cross-linguistic accuracy (88.33%) with minimal language impact (6.67%±0.00%). Resident performance declined significantly under time constraints (from 73.2% to 56.5% adjusted accuracy; Cohen’s d=1.009; <em>P</em><.001), whereas AI systems reported lesser deterioration. Error pattern analysis indicated a moderate correlation between AI and human reasoning (r=0.666; <em>P</em><.001). Residents exhibited systematic progression from year 1 (44.7%) to year 5 (87.1%). Integration analysis found variable benefits across training levels, with maximum enhancement in early-career residents (+29.7%; <em>P</em><.001).</div></div><div><h3>Conclusion</h3><div>High-performing AI LLMs reported strong diagnostic accuracy and resilience under linguistic and temporal pressures. These findings suggest that AI-enhanced decision-making may offer particular benefits in obstetrics and gynecology training programs, especially for junior residents, by improving diagnostic consistency and potentially reducing cognitive load in time-sensitive clinical settings.</div></div>","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"3 2","pages":"Article 100206"},"PeriodicalIF":0.0,"publicationDate":"2025-03-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143697951","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-03-08DOI: 10.1016/j.mcpdig.2025.100208
{"title":"Corrigendum to “Experience With an Optical Character Recognition Search Application for Review of Outside Medical Records”","authors":"","doi":"10.1016/j.mcpdig.2025.100208","DOIUrl":"10.1016/j.mcpdig.2025.100208","url":null,"abstract":"","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"3 2","pages":"Article 100208"},"PeriodicalIF":0.0,"publicationDate":"2025-03-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143697824","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
To systematically evaluate the effectiveness and methodologic approaches of natural language processing (NLP) techniques for early detection of cognitive decline through speech and language analysis.
Methods
We conducted a comprehensive search of 8 databases from inception through August 31, 2024, following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Studies were included if they used NLP techniques to analyze speech or language data for detecting cognitive impairment and reported diagnostic accuracy metrics. Two independent reviewers (R.S. and A.B.) screened articles and extracted data on study characteristics, NLP methods, and outcomes.
Results
Of 23,562 records identified, 51 studies met inclusion criteria, involving 17,340 participants (mean age, 72.4 years). Combined linguistic and acoustic approaches achieved the highest diagnostic accuracy (average 87%; area under the curve [AUC], 0.89) compared with linguistic-only (83%; AUC, 0.85) or acoustic-only approaches (80%; AUC, 0.82). Lexical diversity, syntactic complexity, and semantic coherence were consistently strong predictors across cognitive conditions. Picture description tasks were most common (n=21), followed by spontaneous speech (n=15) and story recall (n=8). Crosslinguistic applicability was found across 8 languages, although language-specific adaptations were necessary. Longitudinal studies (n=9) reported potential for early detection but were limited by smaller sample sizes (average n=159) compared with cross-sectional studies (n=42; average n=274).
Conclusion
Natural language processing techniques show promising diagnostic accuracy for detecting cognitive impairment across multiple languages and clinical contexts. Although combined linguistic-acoustic approaches appear most effective, methodologic heterogeneity and small sample sizes in existing studies suggest the need for larger, standardized investigations to establish clinical utility.
{"title":"A Systematic Review of Natural Language Processing Techniques for Early Detection of Cognitive Impairment","authors":"Ravi Shankar PhD , Anjali Bundele MPH , Amartya Mukhopadhyay FRCP","doi":"10.1016/j.mcpdig.2025.100205","DOIUrl":"10.1016/j.mcpdig.2025.100205","url":null,"abstract":"<div><h3>Objective</h3><div>To systematically evaluate the effectiveness and methodologic approaches of natural language processing (NLP) techniques for early detection of cognitive decline through speech and language analysis.</div></div><div><h3>Methods</h3><div>We conducted a comprehensive search of 8 databases from inception through August 31, 2024, following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Studies were included if they used NLP techniques to analyze speech or language data for detecting cognitive impairment and reported diagnostic accuracy metrics. Two independent reviewers (R.S. and A.B.) screened articles and extracted data on study characteristics, NLP methods, and outcomes.</div></div><div><h3>Results</h3><div>Of 23,562 records identified, 51 studies met inclusion criteria, involving 17,340 participants (mean age, 72.4 years). Combined linguistic and acoustic approaches achieved the highest diagnostic accuracy (average 87%; area under the curve [AUC], 0.89) compared with linguistic-only (83%; AUC, 0.85) or acoustic-only approaches (80%; AUC, 0.82). Lexical diversity, syntactic complexity, and semantic coherence were consistently strong predictors across cognitive conditions. Picture description tasks were most common (n=21), followed by spontaneous speech (n=15) and story recall (n=8). Crosslinguistic applicability was found across 8 languages, although language-specific adaptations were necessary. Longitudinal studies (n=9) reported potential for early detection but were limited by smaller sample sizes (average n=159) compared with cross-sectional studies (n=42; average n=274).</div></div><div><h3>Conclusion</h3><div>Natural language processing techniques show promising diagnostic accuracy for detecting cognitive impairment across multiple languages and clinical contexts. Although combined linguistic-acoustic approaches appear most effective, methodologic heterogeneity and small sample sizes in existing studies suggest the need for larger, standardized investigations to establish clinical utility.</div></div>","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"3 2","pages":"Article 100205"},"PeriodicalIF":0.0,"publicationDate":"2025-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143697822","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-03-03DOI: 10.1016/j.mcpdig.2025.100201
Roland Barge PhD , Patrick Floody MBA
Objective
To define potential participant and site perceptions of decentralized clinical trials (DCTs).
Participants and Methods
Two qualitative surveys were conducted between January 2022 and August 2022 to assess current awareness of, and perceptions about, DCTs. The first survey received 141 responses from staff at our clinical trial sites; the second survey received 481 responses from US-based healthy individuals or those living with an illness.
Results
There was a difference in perceptions and willingness between participants and sites toward DCTs. Participants expressed more comfort with hybrid and fully remote trials than did the sites. Site staff were more concerned and less trusting than participants of DCTs; participants’ main concerns were regarding practicality and medical safety, whereas the focus for sites was on burden, trust, and security. Both sites and participants expressed confidence in fully remote clinical study activities when they have appropriate support; sites were less tolerant of fully remote clinical study activities if professional support was not provided. Overall, sites were more willing to manage the use of DCT-related technologies than were participants. It is highly likely that participants’ willingness to manage DCT technologies relates to the perceived burden of use (ie, willingness decreases as burden or impact on daily life increases). Sponsors, contract research organizations, and DCT vendors generally had positive views on DCTs. However, different stakeholders had different concerns.
Conclusion
These results highlight the need for collaborative research and development of DCTs, as well as a clear DCT framework and regulatory guidance.
{"title":"Comparison of Participant and Site Perceptions of Decentralized Clinical Trials in the USA","authors":"Roland Barge PhD , Patrick Floody MBA","doi":"10.1016/j.mcpdig.2025.100201","DOIUrl":"10.1016/j.mcpdig.2025.100201","url":null,"abstract":"<div><h3>Objective</h3><div>To define potential participant and site perceptions of decentralized clinical trials (DCTs).</div></div><div><h3>Participants and Methods</h3><div>Two qualitative surveys were conducted between January 2022 and August 2022 to assess current awareness of, and perceptions about, DCTs. The first survey received 141 responses from staff at our clinical trial sites; the second survey received 481 responses from US-based healthy individuals or those living with an illness.</div></div><div><h3>Results</h3><div>There was a difference in perceptions and willingness between participants and sites toward DCTs. Participants expressed more comfort with hybrid and fully remote trials than did the sites. Site staff were more concerned and less trusting than participants of DCTs; participants’ main concerns were regarding practicality and medical safety, whereas the focus for sites was on burden, trust, and security. Both sites and participants expressed confidence in fully remote clinical study activities when they have appropriate support; sites were less tolerant of fully remote clinical study activities if professional support was not provided. Overall, sites were more willing to manage the use of DCT-related technologies than were participants. It is highly likely that participants’ willingness to manage DCT technologies relates to the perceived burden of use (ie, willingness decreases as burden or impact on daily life increases). Sponsors, contract research organizations, and DCT vendors generally had positive views on DCTs. However, different stakeholders had different concerns.</div></div><div><h3>Conclusion</h3><div>These results highlight the need for collaborative research and development of DCTs, as well as a clear DCT framework and regulatory guidance.</div></div>","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"3 2","pages":"Article 100201"},"PeriodicalIF":0.0,"publicationDate":"2025-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143696654","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-03-03DOI: 10.1016/j.mcpdig.2025.100203
Rebecca J. Calthorpe BMBS , Hisham A. Saumtally MBChB , Laura M. Howells PhD , Natalie J. Goodchild BA (Hons) , Bethinn C. Evans MSc , Zoe Elliott , Bu’Hussain Hayee PhD , Siobhán B. Carr MBBS , Caroline M. Elston MBBS , Alexander A.R. Horsley PhD , Daniel G. Peckham DM , Helen L. Barr PhD , Giles A.D. Major PhD , Iain D. Stewart PhD , Kim S. Thomas , Alan R. Smyth MD
Objective
To develop a cystic fibrosis (CF)–specific patient-reported outcome measure (PROM) to measure the daily burden of gastrointestinal symptoms for people with cystic fibrosis (pwCF) aged 12 years and older and address the lack of validated outcome measures for gastrointestinal symptoms in CF.
Patients and Methods
CF Tummy Tracker was developed through a 5-stage approach in accordance with regulatory guidance. This included development and refinement of a conceptual framework; item generation; refinement; reduction; selection; and initial PROM testing. A mixed-methods approach, consisting of expert panel discussions, a focus group, interviews, and an online survey, was used. In initial testing, participants completed the PROM daily for 14 days via a smartphone application. This study was performed from March 14, 2022, December 12, 2023.
Results
The CF community were involved throughout the development via a focus group (n=7 pwCF), interviews (n=11 pwCF), and an online survey (n=180 pwCF). A formative model was confirmed for the PROM. The final PROM, CF Tummy Tracker, consists of 10 items capturing gastrointestinal symptom burden, tested in 151 pwCF. The PROM reported no floor or ceiling effects, high test–retest reliability (intra-class correlation coefficient=0.94), and strong correlation with the anchor question.
Conclusion
CF Tummy Tracker aims to address the gap in validated CF-specific PROMs for daily completion. Further testing of the psychometric properties of the PROM are planned in a new patient cohort to validate its use in clinical trials and support its use in both electronic and paper formats to increase accessibility.
{"title":"CF Tummy Tracker: A Cystic Fibrosis–Specific Patient-Reported Outcome Measure for Daily Gastrointestinal Symptom Burden","authors":"Rebecca J. Calthorpe BMBS , Hisham A. Saumtally MBChB , Laura M. Howells PhD , Natalie J. Goodchild BA (Hons) , Bethinn C. Evans MSc , Zoe Elliott , Bu’Hussain Hayee PhD , Siobhán B. Carr MBBS , Caroline M. Elston MBBS , Alexander A.R. Horsley PhD , Daniel G. Peckham DM , Helen L. Barr PhD , Giles A.D. Major PhD , Iain D. Stewart PhD , Kim S. Thomas , Alan R. Smyth MD","doi":"10.1016/j.mcpdig.2025.100203","DOIUrl":"10.1016/j.mcpdig.2025.100203","url":null,"abstract":"<div><h3>Objective</h3><div>To develop a cystic fibrosis (CF)–specific patient-reported outcome measure (PROM) to measure the daily burden of gastrointestinal symptoms for people with cystic fibrosis (pwCF) aged 12 years and older and address the lack of validated outcome measures for gastrointestinal symptoms in CF.</div></div><div><h3>Patients and Methods</h3><div>CF Tummy Tracker was developed through a 5-stage approach in accordance with regulatory guidance. This included development and refinement of a conceptual framework; item generation; refinement; reduction; selection; and initial PROM testing. A mixed-methods approach, consisting of expert panel discussions, a focus group, interviews, and an online survey, was used. In initial testing, participants completed the PROM daily for 14 days via a smartphone application. This study was performed from March 14, 2022, December 12, 2023.</div></div><div><h3>Results</h3><div>The CF community were involved throughout the development via a focus group (n=7 pwCF), interviews (n=11 pwCF), and an online survey (n=180 pwCF). A formative model was confirmed for the PROM. The final PROM, CF Tummy Tracker, consists of 10 items capturing gastrointestinal symptom burden, tested in 151 pwCF. The PROM reported no floor or ceiling effects, high test–retest reliability (intra-class correlation coefficient=0.94), and strong correlation with the anchor question.</div></div><div><h3>Conclusion</h3><div>CF Tummy Tracker aims to address the gap in validated CF-specific PROMs for daily completion. Further testing of the psychometric properties of the PROM are planned in a new patient cohort to validate its use in clinical trials and support its use in both electronic and paper formats to increase accessibility.</div></div>","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"3 2","pages":"Article 100203"},"PeriodicalIF":0.0,"publicationDate":"2025-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143680721","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-03-01DOI: 10.1016/j.mcpdig.2025.100198
Yuexing Hao MS , Jason Holmes PhD , Jared Hobson MD , Alexandra Bennett MD , Elizabeth L. McKone MD , Daniel K. Ebner MD , David M. Routman MD , Satomi Shiraishi MD , Samir H. Patel MD , Nathan Y. Yu MD , Chris L. Hallemeier MD , Brooke E. Ball MSN , Mark Waddle MD , Wei Liu PhD
Objective
To evaluate the effectiveness of RadOnc-generative pretrained transformer (GPT), a GPT-4 based large language model, in assisting with in-basket message response generation for prostate cancer treatment, with the goal of reducing the workload and time on clinical care teams while maintaining response quality.
Patients and Methods
RadOnc-GPT was integrated with electronic health records from both Mayo Clinic-wide databases and a radiation-oncology-specific database. The model was evaluated on 158 previously recorded in-basket message interactions, selected from 90 patients with nonmetastatic prostate cancer from the Mayo Clinic Department of Radiation Oncology in-basket message database in the calendar years 2022-2024. Quantitative natural language processing analysis and 2 grading studies, conducted by 5 clinicians and 4 nurses, were used to assess RadOnc-GPT’s responses. Three primary clinicians independently graded all messages, whereas a fourth senior clinician reviewed 41 responses with relevant discrepancies, and a fifth senior clinician evaluated 2 additional responses. The grading focused on 5 key areas: completeness, correctness, clarity, empathy, and editing time. The grading study was performed from July 20, 2024 to December 15, 2024.
Results
The RadOnc-GPT slightly outperformed the clinical care team in empathy, whereas achieving comparable scores with the clinical care team in completeness, correctness, and clarity. Five clinician graders identified key limitations in RadOnc-GPT’s responses, such as lack of context, insufficient domain-specific knowledge, inability to perform essential meta-tasks, and hallucination. It was estimated that RadOnc-GPT could save an average of 5.2 minutes per message for nurses and 2.4 minutes for clinicians, from reading the inquiry to sending the response.
Conclusion
RadOnc-GPT has the potential to considerably reduce the workload of clinical care teams by generating high-quality, timely responses for in-basket message interactions. This could lead to improved efficiency in health care workflows and reduced costs while maintaining or enhancing the quality of communication between patients and health care providers.
{"title":"Retrospective Comparative Analysis of Prostate Cancer In-Basket Messages: Responses From Closed-Domain Large Language Models Versus Clinical Teams","authors":"Yuexing Hao MS , Jason Holmes PhD , Jared Hobson MD , Alexandra Bennett MD , Elizabeth L. McKone MD , Daniel K. Ebner MD , David M. Routman MD , Satomi Shiraishi MD , Samir H. Patel MD , Nathan Y. Yu MD , Chris L. Hallemeier MD , Brooke E. Ball MSN , Mark Waddle MD , Wei Liu PhD","doi":"10.1016/j.mcpdig.2025.100198","DOIUrl":"10.1016/j.mcpdig.2025.100198","url":null,"abstract":"<div><h3>Objective</h3><div>To evaluate the effectiveness of RadOnc-generative pretrained transformer (GPT), a GPT-4 based large language model, in assisting with in-basket message response generation for prostate cancer treatment, with the goal of reducing the workload and time on clinical care teams while maintaining response quality.</div></div><div><h3>Patients and Methods</h3><div>RadOnc-GPT was integrated with electronic health records from both Mayo Clinic-wide databases and a radiation-oncology-specific database. The model was evaluated on 158 previously recorded in-basket message interactions, selected from 90 patients with nonmetastatic prostate cancer from the Mayo Clinic Department of Radiation Oncology in-basket message database in the calendar years 2022-2024. Quantitative natural language processing analysis and 2 grading studies, conducted by 5 clinicians and 4 nurses, were used to assess RadOnc-GPT’s responses. Three primary clinicians independently graded all messages, whereas a fourth senior clinician reviewed 41 responses with relevant discrepancies, and a fifth senior clinician evaluated 2 additional responses. The grading focused on 5 key areas: completeness, correctness, clarity, empathy, and editing time. The grading study was performed from July 20, 2024 to December 15, 2024.</div></div><div><h3>Results</h3><div>The RadOnc-GPT slightly outperformed the clinical care team in empathy, whereas achieving comparable scores with the clinical care team in completeness, correctness, and clarity. Five clinician graders identified key limitations in RadOnc-GPT’s responses, such as lack of context, insufficient domain-specific knowledge, inability to perform essential meta-tasks, and hallucination. It was estimated that RadOnc-GPT could save an average of 5.2 minutes per message for nurses and 2.4 minutes for clinicians, from reading the inquiry to sending the response.</div></div><div><h3>Conclusion</h3><div>RadOnc-GPT has the potential to considerably reduce the workload of clinical care teams by generating high-quality, timely responses for in-basket message interactions. This could lead to improved efficiency in health care workflows and reduced costs while maintaining or enhancing the quality of communication between patients and health care providers.</div></div>","PeriodicalId":74127,"journal":{"name":"Mayo Clinic Proceedings. Digital health","volume":"3 1","pages":"Article 100198"},"PeriodicalIF":0.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143579568","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号