Pub Date : 2025-01-13eCollection Date: 2025-01-01DOI: 10.1371/journal.pdig.0000534
Dominique Vincent-Genod, Sylvain Roche, Aurélie Barrière, Capucine de Lattre, Marie Tinat, Eelke Venema, Emmeline Lagrange, Adriana Gomes Lisboa de Souza, Guillaume Thomann, Justine Coton, Vincent Gautheron, Léonard Féasson, Pascal Rippert, Carole Vuillerot
Among the 32 items of the Motor Function Measure scale, 3 concern the assessment of hand function on a paper-based support. Their characteristics make it possible to envisage the use of a tablet instead of the original paper-based support for their completion. This would then make it possible to automate the score to reduce intra- and inter-individual variability. The main objective of the present study was to validate the digital completion of items 18, 19, and 22 by measuring the agreement of the scores obtained using a digital tablet with those obtained using the original paper-based support in children and adults with various neuromuscular diseases (NMD). The secondary objective is to calibrate an algorithm for the automatic items scoring.
Methods: Ninety-eight subjects aged 5 to 60 years with a confirmed NMD completed MFM items 18, 19, and 22 both on a paper support and a digital tablet.
Results: The median age of included subjects was 16.2 years. Agreement between scores as assessed using the weighted Kappa coefficient was almost perfect for the scores of items 18 and 22 (K = 0.93, and 0.95, respectively) and substantial for item 19 (K = 0.70). In all cases of disagreement, the difference was of 1 point. The most frequent disagreement concerned item 19; mainly in the direction of a scoring of 1 point less on the tablet. An automatic analysis algorithm was tested on 82 recordings to suggest improvements.
Conclusion: The switch from original paper-based support to the tablet results in minimal and acceptable differences, and maintains a valid and reproducible measure of the 3 items. The developed algorithm for automatic scoring appears feasible with the perspective to include them in a digital application that will make it easier to monitor patients.
{"title":"Use of assistive technology to assess distal motor function in subjects with neuromuscular disease.","authors":"Dominique Vincent-Genod, Sylvain Roche, Aurélie Barrière, Capucine de Lattre, Marie Tinat, Eelke Venema, Emmeline Lagrange, Adriana Gomes Lisboa de Souza, Guillaume Thomann, Justine Coton, Vincent Gautheron, Léonard Féasson, Pascal Rippert, Carole Vuillerot","doi":"10.1371/journal.pdig.0000534","DOIUrl":"10.1371/journal.pdig.0000534","url":null,"abstract":"<p><p>Among the 32 items of the Motor Function Measure scale, 3 concern the assessment of hand function on a paper-based support. Their characteristics make it possible to envisage the use of a tablet instead of the original paper-based support for their completion. This would then make it possible to automate the score to reduce intra- and inter-individual variability. The main objective of the present study was to validate the digital completion of items 18, 19, and 22 by measuring the agreement of the scores obtained using a digital tablet with those obtained using the original paper-based support in children and adults with various neuromuscular diseases (NMD). The secondary objective is to calibrate an algorithm for the automatic items scoring.</p><p><strong>Design: </strong>Prospective, multicentre, non-interventional study.</p><p><strong>Methods: </strong>Ninety-eight subjects aged 5 to 60 years with a confirmed NMD completed MFM items 18, 19, and 22 both on a paper support and a digital tablet.</p><p><strong>Results: </strong>The median age of included subjects was 16.2 years. Agreement between scores as assessed using the weighted Kappa coefficient was almost perfect for the scores of items 18 and 22 (K = 0.93, and 0.95, respectively) and substantial for item 19 (K = 0.70). In all cases of disagreement, the difference was of 1 point. The most frequent disagreement concerned item 19; mainly in the direction of a scoring of 1 point less on the tablet. An automatic analysis algorithm was tested on 82 recordings to suggest improvements.</p><p><strong>Conclusion: </strong>The switch from original paper-based support to the tablet results in minimal and acceptable differences, and maintains a valid and reproducible measure of the 3 items. The developed algorithm for automatic scoring appears feasible with the perspective to include them in a digital application that will make it easier to monitor patients.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 1","pages":"e0000534"},"PeriodicalIF":0.0,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11729976/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142980867","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-13eCollection Date: 2025-01-01DOI: 10.1371/journal.pdig.0000670
Sara Mesquita, Lília Perfeito, Daniela Paolotti, Joana Gonçalves-Sá
Epidemiology and Public Health have increasingly relied on structured and unstructured data, collected inside and outside of typical health systems, to study, identify, and mitigate diseases at the population level. Focusing on infectious diseases, we review the state of Digital Epidemiology at the beginning of 2020 and how it changed after the COVID-19 pandemic, in both nature and breadth. We argue that Epidemiology's progressive use of data generated outside of clinical and public health systems creates several technical challenges, particularly in carrying specific biases that are almost impossible to correct for a priori. Using a statistical perspective, we discuss how a definition of Digital Epidemiology that emphasizes "data-type" instead of "data-source," may be more operationally useful, by clarifying key methodological differences and gaps. Therefore, we briefly describe some of the possible biases arising from varied collection methods and sources, and offer some recommendations to better explore the potential of Digital Epidemiology, particularly on how to help reduce inequity.
{"title":"Epidemiological methods in transition: Minimizing biases in classical and digital approaches.","authors":"Sara Mesquita, Lília Perfeito, Daniela Paolotti, Joana Gonçalves-Sá","doi":"10.1371/journal.pdig.0000670","DOIUrl":"10.1371/journal.pdig.0000670","url":null,"abstract":"<p><p>Epidemiology and Public Health have increasingly relied on structured and unstructured data, collected inside and outside of typical health systems, to study, identify, and mitigate diseases at the population level. Focusing on infectious diseases, we review the state of Digital Epidemiology at the beginning of 2020 and how it changed after the COVID-19 pandemic, in both nature and breadth. We argue that Epidemiology's progressive use of data generated outside of clinical and public health systems creates several technical challenges, particularly in carrying specific biases that are almost impossible to correct for a priori. Using a statistical perspective, we discuss how a definition of Digital Epidemiology that emphasizes \"data-type\" instead of \"data-source,\" may be more operationally useful, by clarifying key methodological differences and gaps. Therefore, we briefly describe some of the possible biases arising from varied collection methods and sources, and offer some recommendations to better explore the potential of Digital Epidemiology, particularly on how to help reduce inequity.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 1","pages":"e0000670"},"PeriodicalIF":0.0,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11730375/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142980853","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Postnatal care refers to the support provided to mothers and their newborns immediately after childbirth and during the first six weeks of life, a period when most maternal and neonatal deaths occur. In the 30 countries studied, nearly 40 percent of women did not receive a postpartum care check-up. This research aims to evaluate and compare the effectiveness of machine learning algorithms in predicting postnatal care utilization in Ethiopia and to identify the key factors involved. The study employs machine learning techniques to analyse secondary data from the 2016 Ethiopian Demographic and Health Survey. It aims to predict postnatal care utilization and identify key predictors via Python software, applying fifteen machine-learning algorithms to a sample of 7,193 women. Feature importance techniques were used to select the top predictors. The models' effectiveness was evaluated using sensitivity, specificity, F1 score, precision, accuracy, and area under the curve. Among the four experiments, tenfold cross-validation with balancing using Synthetic Minority Over-sampling Technique was outperformed. From fifteen models, the MLP Classifier (f1 score = 0.9548, AUC = 0.99), Random Forest Classifier (f1 score = 0.9543, AUC = 0.98), and Bagging Classifier (f1 score = 0.9498, AUC = 0.98) performed excellently, with a strong ability to differentiate between classes. The Region, residence, maternal education, religion, wealth index, health insurance status, and place of delivery are identified as contributing factors that predict postnatal care utilization. This study assessed machine learning models for forecasting postnatal care usage. Ten-fold cross-validation with Synthetic Minority Oversampling Technique produced the best results, emphasizing the significance of addressing class imbalance in healthcare datasets. This approach enhances the accuracy and dependability of predictive models. Key findings reveal regional and socioeconomic factors influencing PNC utilization, which can guide targeted initiatives to improve postnatal care utilization and ultimately enhance maternal and child health.
{"title":"Performance evaluation and comparative analysis of different machine learning algorithms in predicting postnatal care utilization: Evidence from the ethiopian demographic and health survey 2016.","authors":"Daniel Niguse Mamo, Agmasie Damtew Walle, Eden Ketema Woldekidan, Jibril Bashir Adem, Yosef Haile Gebremariam, Meron Asmamaw Alemayehu, Ermias Bekele Enyew, Shimels Derso Kebede","doi":"10.1371/journal.pdig.0000707","DOIUrl":"10.1371/journal.pdig.0000707","url":null,"abstract":"<p><p>Postnatal care refers to the support provided to mothers and their newborns immediately after childbirth and during the first six weeks of life, a period when most maternal and neonatal deaths occur. In the 30 countries studied, nearly 40 percent of women did not receive a postpartum care check-up. This research aims to evaluate and compare the effectiveness of machine learning algorithms in predicting postnatal care utilization in Ethiopia and to identify the key factors involved. The study employs machine learning techniques to analyse secondary data from the 2016 Ethiopian Demographic and Health Survey. It aims to predict postnatal care utilization and identify key predictors via Python software, applying fifteen machine-learning algorithms to a sample of 7,193 women. Feature importance techniques were used to select the top predictors. The models' effectiveness was evaluated using sensitivity, specificity, F1 score, precision, accuracy, and area under the curve. Among the four experiments, tenfold cross-validation with balancing using Synthetic Minority Over-sampling Technique was outperformed. From fifteen models, the MLP Classifier (f1 score = 0.9548, AUC = 0.99), Random Forest Classifier (f1 score = 0.9543, AUC = 0.98), and Bagging Classifier (f1 score = 0.9498, AUC = 0.98) performed excellently, with a strong ability to differentiate between classes. The Region, residence, maternal education, religion, wealth index, health insurance status, and place of delivery are identified as contributing factors that predict postnatal care utilization. This study assessed machine learning models for forecasting postnatal care usage. Ten-fold cross-validation with Synthetic Minority Oversampling Technique produced the best results, emphasizing the significance of addressing class imbalance in healthcare datasets. This approach enhances the accuracy and dependability of predictive models. Key findings reveal regional and socioeconomic factors influencing PNC utilization, which can guide targeted initiatives to improve postnatal care utilization and ultimately enhance maternal and child health.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 1","pages":"e0000707"},"PeriodicalIF":0.0,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11717314/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142960064","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-09eCollection Date: 2025-01-01DOI: 10.1371/journal.pdig.0000701
Madison Taylor, Denise Ng, Kaylen J Pfisterer, Joseph A Cafazzo, Diana Sherifali
The multicomponent Remission Evaluation of Medical Interventions in T2D (REMIT) program has shown reduction of hazard of diabetes relapse by 34-43%, but could benefit from improved ability to scale, spread, and sustain it. This study explored, at the conceptualization phase, patient and health coach perspectives on the acceptability, adoption, feasibility, and appropriateness of a digital REMIT adaptation (diabetes technology enabled coaching (DTEC)). Twelve semi-structured interviews were conducted with patients (n = 6) and health coaches (n = 6) to explore their experiences with the REMIT study, opportunities for virtualisation, and a cognitive walkthrough of solution concepts. Transcripts were analyzed both inductively and deductively to allow for organic themes to emerge and to position these themes around the constructs of acceptability, adoption, feasibility, and appropriateness while allowing new codes to emerge for discussion. Participants saw value in DTEC as: an opportunity to facilitate and extend REMIT support; a convenient, efficient, and scalable concept (acceptability); having potential to motivate through connecting behaviours to outcomes (adoption); an opportunity for lower-effort demands for sustained use (feasibility). Participants also highlighted important considerations to ensure DTEC could provide compassionate insights and support automated data entry (appropriateness). Several considerations regarding equitable access were raised and warrant further consideration including: provision of technology, training to support technology literacy, and the opportunity for DTEC to support and improve health literacy. As such, DTEC may act as a moderator that can enhance or diminish access which affects who can benefit. Provided equity considerations are addressed, DTEC has the potential to address previous shortcomings of the conventional REMIT program.
{"title":"The value of diabetes technology enabled coaching (DTEC) to support remission evaluation of medical interventions in T2D: Patient and health coach perspectives.","authors":"Madison Taylor, Denise Ng, Kaylen J Pfisterer, Joseph A Cafazzo, Diana Sherifali","doi":"10.1371/journal.pdig.0000701","DOIUrl":"10.1371/journal.pdig.0000701","url":null,"abstract":"<p><p>The multicomponent Remission Evaluation of Medical Interventions in T2D (REMIT) program has shown reduction of hazard of diabetes relapse by 34-43%, but could benefit from improved ability to scale, spread, and sustain it. This study explored, at the conceptualization phase, patient and health coach perspectives on the acceptability, adoption, feasibility, and appropriateness of a digital REMIT adaptation (diabetes technology enabled coaching (DTEC)). Twelve semi-structured interviews were conducted with patients (n = 6) and health coaches (n = 6) to explore their experiences with the REMIT study, opportunities for virtualisation, and a cognitive walkthrough of solution concepts. Transcripts were analyzed both inductively and deductively to allow for organic themes to emerge and to position these themes around the constructs of acceptability, adoption, feasibility, and appropriateness while allowing new codes to emerge for discussion. Participants saw value in DTEC as: an opportunity to facilitate and extend REMIT support; a convenient, efficient, and scalable concept (acceptability); having potential to motivate through connecting behaviours to outcomes (adoption); an opportunity for lower-effort demands for sustained use (feasibility). Participants also highlighted important considerations to ensure DTEC could provide compassionate insights and support automated data entry (appropriateness). Several considerations regarding equitable access were raised and warrant further consideration including: provision of technology, training to support technology literacy, and the opportunity for DTEC to support and improve health literacy. As such, DTEC may act as a moderator that can enhance or diminish access which affects who can benefit. Provided equity considerations are addressed, DTEC has the potential to address previous shortcomings of the conventional REMIT program.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 1","pages":"e0000701"},"PeriodicalIF":0.0,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11717255/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142960067","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-09eCollection Date: 2025-01-01DOI: 10.1371/journal.pdig.0000464
John J L Jacobs, Inés Beekers, Inge Verkouter, Levi B Richards, Alexandra Vegelien, Lizan D Bloemsma, Vera A M C Bongaerts, Jacqueline Cloos, Frederik Erkens, Patrycja Gradowska, Simon Hort, Michael Hudecek, Manel Juan, Anke H Maitland-van der Zee, Sergio Navarro-Velázquez, Lok Lam Ngai, Qasim A Rafiq, Carmen Sanges, Jesse Tettero, Hendrikus J A van Os, Rimke C Vos, Yolanda de Wit, Steven van Dijk
Precision, or personalised medicine has advanced requirements for medical data management systems (MedDMSs). MedDMS for precision medicine should be able to process hundreds of parameters from multiple sites, be adaptable while remaining in sync at multiple locations, real-time syncing to analytics and be compliant with international privacy legislation. This paper describes the LogiqSuite software solution, aimed to support a precision medicine solution at the patient care (LogiqCare), research (LogiqScience) and data science (LogiqAnalytics) level. LogiqSuite is certified and compliant with international medical data and privacy legislations. This paper evaluates a MedDMS in five types of use cases for precision medicine, ranging from data collection to algorithm development and from implementation to integration with real-world data. The MedDMS is evaluated in seven precision medicine data science projects in prehospital triage, cardiovascular disease, pulmonology, and oncology. The P4O2 consortium uses the MedDMS as an electronic case report form (eCRF) that allows real-time data management and analytics in long covid and pulmonary diseases. In an acute myeloid leukaemia, study data from different sources were integrated to facilitate easy descriptive analytics for various research questions. In the AIDPATH project, LogiqCare is used to process patient data, while LogiqScience is used for pseudonymous CAR-T cell production for cancer treatment. In both these oncological projects the data in LogiqAnalytics is also used to facilitate machine learning to develop new prediction models for clinical-decision support (CDS). The MedDMS is also evaluated for real-time recording of CDS data from U-Prevent for cardiovascular risk management and from the Stroke Triage App for prehospital triage. The MedDMS is discussed in relation to other solutions for privacy-by-design, integrated data stewardship and real-time data analytics in precision medicine. LogiqSuite is used for multi-centre research study data registrations and monitoring, data analytics in interdisciplinary consortia, design of new machine learning / artificial intelligence (AI) algorithms, development of new or updated prediction models, integration of care with advanced therapy production, and real-world data monitoring in using CDS tools. The integrated MedDMS application supports data management for care and research in precision medicine.
{"title":"A data management system for precision medicine.","authors":"John J L Jacobs, Inés Beekers, Inge Verkouter, Levi B Richards, Alexandra Vegelien, Lizan D Bloemsma, Vera A M C Bongaerts, Jacqueline Cloos, Frederik Erkens, Patrycja Gradowska, Simon Hort, Michael Hudecek, Manel Juan, Anke H Maitland-van der Zee, Sergio Navarro-Velázquez, Lok Lam Ngai, Qasim A Rafiq, Carmen Sanges, Jesse Tettero, Hendrikus J A van Os, Rimke C Vos, Yolanda de Wit, Steven van Dijk","doi":"10.1371/journal.pdig.0000464","DOIUrl":"10.1371/journal.pdig.0000464","url":null,"abstract":"<p><p>Precision, or personalised medicine has advanced requirements for medical data management systems (MedDMSs). MedDMS for precision medicine should be able to process hundreds of parameters from multiple sites, be adaptable while remaining in sync at multiple locations, real-time syncing to analytics and be compliant with international privacy legislation. This paper describes the LogiqSuite software solution, aimed to support a precision medicine solution at the patient care (LogiqCare), research (LogiqScience) and data science (LogiqAnalytics) level. LogiqSuite is certified and compliant with international medical data and privacy legislations. This paper evaluates a MedDMS in five types of use cases for precision medicine, ranging from data collection to algorithm development and from implementation to integration with real-world data. The MedDMS is evaluated in seven precision medicine data science projects in prehospital triage, cardiovascular disease, pulmonology, and oncology. The P4O2 consortium uses the MedDMS as an electronic case report form (eCRF) that allows real-time data management and analytics in long covid and pulmonary diseases. In an acute myeloid leukaemia, study data from different sources were integrated to facilitate easy descriptive analytics for various research questions. In the AIDPATH project, LogiqCare is used to process patient data, while LogiqScience is used for pseudonymous CAR-T cell production for cancer treatment. In both these oncological projects the data in LogiqAnalytics is also used to facilitate machine learning to develop new prediction models for clinical-decision support (CDS). The MedDMS is also evaluated for real-time recording of CDS data from U-Prevent for cardiovascular risk management and from the Stroke Triage App for prehospital triage. The MedDMS is discussed in relation to other solutions for privacy-by-design, integrated data stewardship and real-time data analytics in precision medicine. LogiqSuite is used for multi-centre research study data registrations and monitoring, data analytics in interdisciplinary consortia, design of new machine learning / artificial intelligence (AI) algorithms, development of new or updated prediction models, integration of care with advanced therapy production, and real-world data monitoring in using CDS tools. The integrated MedDMS application supports data management for care and research in precision medicine.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 1","pages":"e0000464"},"PeriodicalIF":0.0,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11717228/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142960062","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-08eCollection Date: 2025-01-01DOI: 10.1371/journal.pdig.0000711
Anna R Van Meter, Michael G Wheaton, Victoria E Cosgrove, Katerina Andreadis, Ronald E Robertson
Generative artificial intelligence (genAI) has potential to improve healthcare by reducing clinician burden and expanding services, among other uses. There is a significant gap between the need for mental health care and available clinicians in the United States-this makes it an attractive target for improved efficiency through genAI. Among the most sensitive mental health topics is suicide, and demand for crisis intervention has grown in recent years. We aimed to evaluate the quality of genAI tool responses to suicide-related queries. We entered 10 suicide-related queries into five genAI tools-ChatGPT 3.5, GPT-4, a version of GPT-4 safe for protected health information, Gemini, and Bing Copilot. The response to each query was coded on seven metrics including presence of a suicide hotline number, content related to evidence-based suicide interventions, supportive content, harmful content. Pooling across tools, most of the responses (79%) were supportive. Only 24% of responses included a crisis hotline number and only 4% included content consistent with evidence-based suicide prevention interventions. Harmful content was rare (5%); all such instances were delivered by Bing Copilot. Our results suggest that genAI developers have taken a very conservative approach to suicide-related content and constrained their models' responses to suggest support-seeking, but little else. Finding balance between providing much needed evidence-based mental health information without introducing excessive risk is within the capabilities of genAI developers. At this nascent stage of integrating genAI tools into healthcare systems, ensuring mental health parity should be the goal of genAI developers and healthcare organizations.
{"title":"The Goldilocks Zone: Finding the right balance of user and institutional risk for suicide-related generative AI queries.","authors":"Anna R Van Meter, Michael G Wheaton, Victoria E Cosgrove, Katerina Andreadis, Ronald E Robertson","doi":"10.1371/journal.pdig.0000711","DOIUrl":"10.1371/journal.pdig.0000711","url":null,"abstract":"<p><p>Generative artificial intelligence (genAI) has potential to improve healthcare by reducing clinician burden and expanding services, among other uses. There is a significant gap between the need for mental health care and available clinicians in the United States-this makes it an attractive target for improved efficiency through genAI. Among the most sensitive mental health topics is suicide, and demand for crisis intervention has grown in recent years. We aimed to evaluate the quality of genAI tool responses to suicide-related queries. We entered 10 suicide-related queries into five genAI tools-ChatGPT 3.5, GPT-4, a version of GPT-4 safe for protected health information, Gemini, and Bing Copilot. The response to each query was coded on seven metrics including presence of a suicide hotline number, content related to evidence-based suicide interventions, supportive content, harmful content. Pooling across tools, most of the responses (79%) were supportive. Only 24% of responses included a crisis hotline number and only 4% included content consistent with evidence-based suicide prevention interventions. Harmful content was rare (5%); all such instances were delivered by Bing Copilot. Our results suggest that genAI developers have taken a very conservative approach to suicide-related content and constrained their models' responses to suggest support-seeking, but little else. Finding balance between providing much needed evidence-based mental health information without introducing excessive risk is within the capabilities of genAI developers. At this nascent stage of integrating genAI tools into healthcare systems, ensuring mental health parity should be the goal of genAI developers and healthcare organizations.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 1","pages":"e0000711"},"PeriodicalIF":0.0,"publicationDate":"2025-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11709298/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142960066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-07eCollection Date: 2025-01-01DOI: 10.1371/journal.pdig.0000601
Sarah Livermon, Audrey Michel, Yiyang Zhang, Kaitlyn Petz, Emma Toner, Mark Rucker, Mehdi Boukhechba, Laura E Barnes, Bethany A Teachman
Anxiety is highly prevalent among college communities, with significant numbers of students, faculty, and staff experiencing severe anxiety symptoms. Digital mental health interventions (DMHIs), including Cognitive Bias Modification for Interpretation (CBM-I), offer promising solutions to enhance access to mental health care, yet there is a critical need to evaluate user experience and acceptability of DMHIs. CBM-I training targets cognitive biases in threat perception, aiming to increase cognitive flexibility by reducing rigid negative thought patterns and encouraging more benign interpretations of ambiguous situations. This study used questionnaire and interview data to gather feedback from users of a mobile application called "Hoos Think Calmly" (HTC), which offers brief CBM-I training doses in response to stressors commonly experienced by students, faculty, and staff at a large public university. Mixed methods were used for triangulation to enhance the validity of the findings. Qualitative data was collected through semi-structured interviews from a subset of participants (n = 22) and analyzed thematically using an inductive framework, revealing five main themes: Effectiveness of the Training Program; Feedback on Training Sessions; Barriers to Using the App; Use Patterns; and Suggestions for Improvement. Additionally, biweekly user experience questionnaires sent to all participants in the active treatment condition (n = 134) during the parent trial showed the most commonly endorsed response (by 43.30% of participants) was that the program was somewhat helpful in reducing or managing their anxiety or stress. There was overall agreement between the quantitative and qualitative findings, indicating that graduate students found it the most effective and relatable, with results being moderately positive but somewhat more mixed for undergraduate students and staff, and least positive for faculty. Findings point to clear avenues to enhance the relatability and acceptability of DMHIs across diverse demographics through increased customization and personalization, which may help guide development of future DMHIs.
{"title":"A mobile intervention to reduce anxiety among university students, faculty, and staff: Mixed methods study on users' experiences.","authors":"Sarah Livermon, Audrey Michel, Yiyang Zhang, Kaitlyn Petz, Emma Toner, Mark Rucker, Mehdi Boukhechba, Laura E Barnes, Bethany A Teachman","doi":"10.1371/journal.pdig.0000601","DOIUrl":"10.1371/journal.pdig.0000601","url":null,"abstract":"<p><p>Anxiety is highly prevalent among college communities, with significant numbers of students, faculty, and staff experiencing severe anxiety symptoms. Digital mental health interventions (DMHIs), including Cognitive Bias Modification for Interpretation (CBM-I), offer promising solutions to enhance access to mental health care, yet there is a critical need to evaluate user experience and acceptability of DMHIs. CBM-I training targets cognitive biases in threat perception, aiming to increase cognitive flexibility by reducing rigid negative thought patterns and encouraging more benign interpretations of ambiguous situations. This study used questionnaire and interview data to gather feedback from users of a mobile application called \"Hoos Think Calmly\" (HTC), which offers brief CBM-I training doses in response to stressors commonly experienced by students, faculty, and staff at a large public university. Mixed methods were used for triangulation to enhance the validity of the findings. Qualitative data was collected through semi-structured interviews from a subset of participants (n = 22) and analyzed thematically using an inductive framework, revealing five main themes: Effectiveness of the Training Program; Feedback on Training Sessions; Barriers to Using the App; Use Patterns; and Suggestions for Improvement. Additionally, biweekly user experience questionnaires sent to all participants in the active treatment condition (n = 134) during the parent trial showed the most commonly endorsed response (by 43.30% of participants) was that the program was somewhat helpful in reducing or managing their anxiety or stress. There was overall agreement between the quantitative and qualitative findings, indicating that graduate students found it the most effective and relatable, with results being moderately positive but somewhat more mixed for undergraduate students and staff, and least positive for faculty. Findings point to clear avenues to enhance the relatability and acceptability of DMHIs across diverse demographics through increased customization and personalization, which may help guide development of future DMHIs.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 1","pages":"e0000601"},"PeriodicalIF":0.0,"publicationDate":"2025-01-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11706487/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142960063","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-07eCollection Date: 2025-01-01DOI: 10.1371/journal.pdig.0000697
Rebecca Blundell, Christine d'Offay, Charles Hand, Daniel Tadmor, Alan Carson, David Gillespie, Matthew Reed, Aimun A B Jamjoom
Individuals who sustain a concussion can experience a range of symptoms which can significantly impact their quality of life and functional outcome. This study aims to understand the nature and recovery trajectories of post-concussion symptomatology by applying an unsupervised machine learning approach to data captured from a digital health intervention (HeadOn). As part of the 35-day program, patients complete a daily symptom diary which rates 8 post-concussion symptoms. Symptom data were analysed using K-means clustering to categorize patients based on their symptom profiles. During the study period, a total of 758 symptom diaries were completed by 84 patients, equating to 6064 individual symptom ratings. Fatigue, sleep disturbance and difficulty concentrating were the most prevalent symptoms reported. A decline in symptom burden was observed over the 35-day period, with physical and emotional symptoms showing early rates of recovery. In a correlation matrix, there were strong positive correlations between low mood and irritability (r = 0.84), and poor memory and difficulty concentrating (r = 0.83). K-means cluster analysis identified three distinct patient clusters based on symptom severity. Cluster 0 (n = 24) had a low symptom burden profile across all the post-concussion symptoms. Cluster 1 (n = 35) had moderate symptom burden but with pronounced fatigue. Cluster 2 (n = 25) had a high symptom burden profile across all the post-concussion symptoms. Reflecting the severity of the clusters, there was a significant relationship between the symptom clusters for both the Rivermead (p = 0.05) and PHQ-9 (p = 0.003) questionnaires at 6-weeks follow-up. By leveraging digital ecological momentary assessments, a rich dataset of daily symptom ratings was captured allowing for the identification of symptom severity clusters. These findings underscore the potential of digital technology and machine learning to enhance our understanding of post-concussion symptomatology and offer a scalable solution to support patients with their recovery.
{"title":"Post-concussion symptom burden and dynamics: Insights from a digital health intervention and machine learning.","authors":"Rebecca Blundell, Christine d'Offay, Charles Hand, Daniel Tadmor, Alan Carson, David Gillespie, Matthew Reed, Aimun A B Jamjoom","doi":"10.1371/journal.pdig.0000697","DOIUrl":"10.1371/journal.pdig.0000697","url":null,"abstract":"<p><p>Individuals who sustain a concussion can experience a range of symptoms which can significantly impact their quality of life and functional outcome. This study aims to understand the nature and recovery trajectories of post-concussion symptomatology by applying an unsupervised machine learning approach to data captured from a digital health intervention (HeadOn). As part of the 35-day program, patients complete a daily symptom diary which rates 8 post-concussion symptoms. Symptom data were analysed using K-means clustering to categorize patients based on their symptom profiles. During the study period, a total of 758 symptom diaries were completed by 84 patients, equating to 6064 individual symptom ratings. Fatigue, sleep disturbance and difficulty concentrating were the most prevalent symptoms reported. A decline in symptom burden was observed over the 35-day period, with physical and emotional symptoms showing early rates of recovery. In a correlation matrix, there were strong positive correlations between low mood and irritability (r = 0.84), and poor memory and difficulty concentrating (r = 0.83). K-means cluster analysis identified three distinct patient clusters based on symptom severity. Cluster 0 (n = 24) had a low symptom burden profile across all the post-concussion symptoms. Cluster 1 (n = 35) had moderate symptom burden but with pronounced fatigue. Cluster 2 (n = 25) had a high symptom burden profile across all the post-concussion symptoms. Reflecting the severity of the clusters, there was a significant relationship between the symptom clusters for both the Rivermead (p = 0.05) and PHQ-9 (p = 0.003) questionnaires at 6-weeks follow-up. By leveraging digital ecological momentary assessments, a rich dataset of daily symptom ratings was captured allowing for the identification of symptom severity clusters. These findings underscore the potential of digital technology and machine learning to enhance our understanding of post-concussion symptomatology and offer a scalable solution to support patients with their recovery.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 1","pages":"e0000697"},"PeriodicalIF":0.0,"publicationDate":"2025-01-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11706386/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142960065","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-06eCollection Date: 2025-01-01DOI: 10.1371/journal.pdig.0000704
Anne Alarilla, Neil J Sebire, Josh Keith, Mario Cortina-Borja, Jo Wray, Gwyneth Davies
Patient reported outcome measures (PROMs) capture patients' views of their health status and the use of PROMs as part of standard care of children and young people has the potential to improve communication between patients/carers and clinicians and the quality of care. Electronic systems for the collection of or access to PROMs and integrating PROMs into electronic health records facilitates their implementation in routine care and could help maximise their value. Yet little is known about the technical aspects of implementation including the electronic systems available for collection and capture and how this may influence the value of PROMs in routine care which this scoping review aims to explore. The Joanna Briggs Institute review process was used. Seven databases were searched (Emcare, Embase MEDLINE, APA PsychInfo, Scopus and Web of Science), initially in February 2021 and updated in April 2023. Only studies that mentioned the use of electronic systems for the collection, storage and/or access of PROMs as part of standard care of children and young people in secondary (or tertiary) care settings were included. Data were analysed using frequency counts and thematically mapped using basic content analysis in relation to the research questions. From the 372 studies that were eligible for full text review, 85 studies met the inclusion criteria. The findings show that there is great variability in the electronic platforms used in the collection, storage and access of PROMs resulting in different configurations and fragmented approaches to implementation. There appears to be a lack of consideration on the technical aspects of the implementation such as the accessibility, useability and interoperability of the data collected. Electronic platforms for the collection and capture of PROMs in routine care of CYP is popular, yet, further understanding of the technical considerations in the use of electronic systems for implementation is needed to maximise the potential value and support the scalability of PROMs in routine care.
{"title":"A scoping review of the electronic collection and capture of patient reported outcome measures for children and young people in the hospital setting.","authors":"Anne Alarilla, Neil J Sebire, Josh Keith, Mario Cortina-Borja, Jo Wray, Gwyneth Davies","doi":"10.1371/journal.pdig.0000704","DOIUrl":"https://doi.org/10.1371/journal.pdig.0000704","url":null,"abstract":"<p><p>Patient reported outcome measures (PROMs) capture patients' views of their health status and the use of PROMs as part of standard care of children and young people has the potential to improve communication between patients/carers and clinicians and the quality of care. Electronic systems for the collection of or access to PROMs and integrating PROMs into electronic health records facilitates their implementation in routine care and could help maximise their value. Yet little is known about the technical aspects of implementation including the electronic systems available for collection and capture and how this may influence the value of PROMs in routine care which this scoping review aims to explore. The Joanna Briggs Institute review process was used. Seven databases were searched (Emcare, Embase MEDLINE, APA PsychInfo, Scopus and Web of Science), initially in February 2021 and updated in April 2023. Only studies that mentioned the use of electronic systems for the collection, storage and/or access of PROMs as part of standard care of children and young people in secondary (or tertiary) care settings were included. Data were analysed using frequency counts and thematically mapped using basic content analysis in relation to the research questions. From the 372 studies that were eligible for full text review, 85 studies met the inclusion criteria. The findings show that there is great variability in the electronic platforms used in the collection, storage and access of PROMs resulting in different configurations and fragmented approaches to implementation. There appears to be a lack of consideration on the technical aspects of the implementation such as the accessibility, useability and interoperability of the data collected. Electronic platforms for the collection and capture of PROMs in routine care of CYP is popular, yet, further understanding of the technical considerations in the use of electronic systems for implementation is needed to maximise the potential value and support the scalability of PROMs in routine care.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 1","pages":"e0000704"},"PeriodicalIF":0.0,"publicationDate":"2025-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11703060/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143070165","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-03eCollection Date: 2025-01-01DOI: 10.1371/journal.pdig.0000710
Anna Lea Stark-Blomeier, Stephan Krayter, Christoph Dockweiler
Telerehabilitation is a new form of care that provides digital access to rehabilitative services. However, it places many demands on the users-both patients and therapists. The aim of this study was to determine the requirements and competencies needed for successful usage, identify person- and context-specific differences and develop a competency model. We conducted two cross-sectional online surveys with telerehabilitation patients and therapists from Germany during June-August 2023. The adjusted dataset of 262 patients and 73 therapists was quantitatively analyzed including descriptive and bivariate statistics. Group differences were assessed using t-tests or U-tests. The development of two telerehabilitation competency models was guided by a competency modeling process. The surveys show that patients need to gather program information before program start, follow therapist's instructions, adapt therapy, deal with health problems, as well as motivate and remind oneself during the program. Therapists need to inform and instruct patients, adapt therapy, carry out technical set-up and support, give medical support, guide and monitor patients, give feedback, motivation and reminder, as well as documentation. The competency model for patients includes 23 and the model for therapists 24 core competencies, including various required areas of knowledge, skills, attitudes and experiences. The three most relevant competencies for patients are self-interest in the program, self-awareness and self-management. Also, disease severity, age, and language abilities can enable successful execution. Program type, technology affinity, and age significantly influence the rated relevance of competencies. The three most relevant competencies for therapists are therapeutic-professional skills, medical and telerehabilitation knowledge. The type of therapy practiced and language abilities can enable successful execution. Therapist's age, technology affinity, and job type significantly impact the rated relevance. The models should be applied to develop tailored training formats and support decisions on the selection of suitable therapists and patients for telerehabilitation.
{"title":"Developing a competency model for telerehabilitation therapists and patients: Results of a cross-sectional online survey.","authors":"Anna Lea Stark-Blomeier, Stephan Krayter, Christoph Dockweiler","doi":"10.1371/journal.pdig.0000710","DOIUrl":"https://doi.org/10.1371/journal.pdig.0000710","url":null,"abstract":"<p><p>Telerehabilitation is a new form of care that provides digital access to rehabilitative services. However, it places many demands on the users-both patients and therapists. The aim of this study was to determine the requirements and competencies needed for successful usage, identify person- and context-specific differences and develop a competency model. We conducted two cross-sectional online surveys with telerehabilitation patients and therapists from Germany during June-August 2023. The adjusted dataset of 262 patients and 73 therapists was quantitatively analyzed including descriptive and bivariate statistics. Group differences were assessed using t-tests or U-tests. The development of two telerehabilitation competency models was guided by a competency modeling process. The surveys show that patients need to gather program information before program start, follow therapist's instructions, adapt therapy, deal with health problems, as well as motivate and remind oneself during the program. Therapists need to inform and instruct patients, adapt therapy, carry out technical set-up and support, give medical support, guide and monitor patients, give feedback, motivation and reminder, as well as documentation. The competency model for patients includes 23 and the model for therapists 24 core competencies, including various required areas of knowledge, skills, attitudes and experiences. The three most relevant competencies for patients are self-interest in the program, self-awareness and self-management. Also, disease severity, age, and language abilities can enable successful execution. Program type, technology affinity, and age significantly influence the rated relevance of competencies. The three most relevant competencies for therapists are therapeutic-professional skills, medical and telerehabilitation knowledge. The type of therapy practiced and language abilities can enable successful execution. Therapist's age, technology affinity, and job type significantly impact the rated relevance. The models should be applied to develop tailored training formats and support decisions on the selection of suitable therapists and patients for telerehabilitation.</p>","PeriodicalId":74465,"journal":{"name":"PLOS digital health","volume":"4 1","pages":"e0000710"},"PeriodicalIF":0.0,"publicationDate":"2025-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11698311/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142928893","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}