Dylan Koole, Oscar Shen, Amanda Lans, Tom M. de Groot, J. J. Verlaan, J. H. Schwab
{"title":"开发用于识别健康知识有限患者的机器学习算法。","authors":"Dylan Koole, Oscar Shen, Amanda Lans, Tom M. de Groot, J. J. Verlaan, J. H. Schwab","doi":"10.1111/jep.14248","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Rationale</h3>\n \n <p>Limited health literacy (HL) leads to poor health outcomes, psychological stress, and misutilization of medical resources. Although interventions aimed at improving HL may be effective, identifying patients at risk of limited HL in the clinical workflow is challenging. With machine learning (ML) algorithms based on readily available data, healthcare professionals would be enabled to incorporate HL screening without the need for administering in-person HL screening tools.</p>\n </section>\n \n <section>\n \n <h3> Aims and Objectives</h3>\n \n <p>Develop ML algorithms to identify patients at risk for limited HL in spine patients.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>Between December 2021 and February 2023, consecutive English-speaking patients over the age of 18 and new to an urban academic outpatient spine clinic were approached for participation in a cross-sectional survey study. HL was assessed using the Newest Vital Sign and the scores were divided into limited (0–3) and adequate (4–6) HL. Additional patient characteristics were extracted through a sociodemographic survey and electronic health records. Subsequently, feature selection was performed by random forest algorithms with recursive feature selection and five ML models (stochastic gradient boosting, random forest, Bayes point machine, elastic-net penalized logistic regression, support vector machine) were developed to predict limited HL.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>Seven hundred and fifty-three patients were included for model development, of whom 259 (34.4%) had limited HL. Variables identified for predicting limited HL were age, Area Deprivation Index-national, Social Vulnerability Index, insurance category, Body Mass Index, race, college education, and employment status. The Elastic-Net Penalized Logistic Regression algorithm achieved the best performance with a c-statistic of 0.766, calibration slope/intercept of 1.044/−0.037, and Brier score of 0.179.</p>\n </section>\n \n <section>\n \n <h3> Conclusion</h3>\n \n <p>Elastic-Net Penalized Logistic Regression had the best performance when compared with other ML algorithms with a c-statistic of 0.766, calibration slope/intercept of 1.044/−0.037, and a Brier score of 0.179. Over one-third of patients presenting to an outpatient spine center were found to have limited HL. While this algorithm is far from being used in clinical practice, ML algorithms offer a potential opportunity for identifying patients at risk for limited HL without administering in-person HL assessments. This could possibly enable screening and early intervention to mitigate the potential negative consequences of limited HL without taxing the existing clinical workflow.</p>\n </section>\n </div>","PeriodicalId":15997,"journal":{"name":"Journal of evaluation in clinical practice","volume":"31 1","pages":""},"PeriodicalIF":2.1000,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11582738/pdf/","citationCount":"0","resultStr":"{\"title\":\"Development of Machine Learning Algorithms for Identifying Patients With Limited Health Literacy\",\"authors\":\"Dylan Koole, Oscar Shen, Amanda Lans, Tom M. de Groot, J. J. Verlaan, J. H. Schwab\",\"doi\":\"10.1111/jep.14248\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div>\\n \\n \\n <section>\\n \\n <h3> Rationale</h3>\\n \\n <p>Limited health literacy (HL) leads to poor health outcomes, psychological stress, and misutilization of medical resources. Although interventions aimed at improving HL may be effective, identifying patients at risk of limited HL in the clinical workflow is challenging. With machine learning (ML) algorithms based on readily available data, healthcare professionals would be enabled to incorporate HL screening without the need for administering in-person HL screening tools.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Aims and Objectives</h3>\\n \\n <p>Develop ML algorithms to identify patients at risk for limited HL in spine patients.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Methods</h3>\\n \\n <p>Between December 2021 and February 2023, consecutive English-speaking patients over the age of 18 and new to an urban academic outpatient spine clinic were approached for participation in a cross-sectional survey study. HL was assessed using the Newest Vital Sign and the scores were divided into limited (0–3) and adequate (4–6) HL. Additional patient characteristics were extracted through a sociodemographic survey and electronic health records. Subsequently, feature selection was performed by random forest algorithms with recursive feature selection and five ML models (stochastic gradient boosting, random forest, Bayes point machine, elastic-net penalized logistic regression, support vector machine) were developed to predict limited HL.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Results</h3>\\n \\n <p>Seven hundred and fifty-three patients were included for model development, of whom 259 (34.4%) had limited HL. Variables identified for predicting limited HL were age, Area Deprivation Index-national, Social Vulnerability Index, insurance category, Body Mass Index, race, college education, and employment status. The Elastic-Net Penalized Logistic Regression algorithm achieved the best performance with a c-statistic of 0.766, calibration slope/intercept of 1.044/−0.037, and Brier score of 0.179.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Conclusion</h3>\\n \\n <p>Elastic-Net Penalized Logistic Regression had the best performance when compared with other ML algorithms with a c-statistic of 0.766, calibration slope/intercept of 1.044/−0.037, and a Brier score of 0.179. Over one-third of patients presenting to an outpatient spine center were found to have limited HL. While this algorithm is far from being used in clinical practice, ML algorithms offer a potential opportunity for identifying patients at risk for limited HL without administering in-person HL assessments. This could possibly enable screening and early intervention to mitigate the potential negative consequences of limited HL without taxing the existing clinical workflow.</p>\\n </section>\\n </div>\",\"PeriodicalId\":15997,\"journal\":{\"name\":\"Journal of evaluation in clinical practice\",\"volume\":\"31 1\",\"pages\":\"\"},\"PeriodicalIF\":2.1000,\"publicationDate\":\"2024-11-22\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11582738/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of evaluation in clinical practice\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1111/jep.14248\",\"RegionNum\":4,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"HEALTH CARE SCIENCES & SERVICES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of evaluation in clinical practice","FirstCategoryId":"3","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/jep.14248","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
Development of Machine Learning Algorithms for Identifying Patients With Limited Health Literacy
Rationale
Limited health literacy (HL) leads to poor health outcomes, psychological stress, and misutilization of medical resources. Although interventions aimed at improving HL may be effective, identifying patients at risk of limited HL in the clinical workflow is challenging. With machine learning (ML) algorithms based on readily available data, healthcare professionals would be enabled to incorporate HL screening without the need for administering in-person HL screening tools.
Aims and Objectives
Develop ML algorithms to identify patients at risk for limited HL in spine patients.
Methods
Between December 2021 and February 2023, consecutive English-speaking patients over the age of 18 and new to an urban academic outpatient spine clinic were approached for participation in a cross-sectional survey study. HL was assessed using the Newest Vital Sign and the scores were divided into limited (0–3) and adequate (4–6) HL. Additional patient characteristics were extracted through a sociodemographic survey and electronic health records. Subsequently, feature selection was performed by random forest algorithms with recursive feature selection and five ML models (stochastic gradient boosting, random forest, Bayes point machine, elastic-net penalized logistic regression, support vector machine) were developed to predict limited HL.
Results
Seven hundred and fifty-three patients were included for model development, of whom 259 (34.4%) had limited HL. Variables identified for predicting limited HL were age, Area Deprivation Index-national, Social Vulnerability Index, insurance category, Body Mass Index, race, college education, and employment status. The Elastic-Net Penalized Logistic Regression algorithm achieved the best performance with a c-statistic of 0.766, calibration slope/intercept of 1.044/−0.037, and Brier score of 0.179.
Conclusion
Elastic-Net Penalized Logistic Regression had the best performance when compared with other ML algorithms with a c-statistic of 0.766, calibration slope/intercept of 1.044/−0.037, and a Brier score of 0.179. Over one-third of patients presenting to an outpatient spine center were found to have limited HL. While this algorithm is far from being used in clinical practice, ML algorithms offer a potential opportunity for identifying patients at risk for limited HL without administering in-person HL assessments. This could possibly enable screening and early intervention to mitigate the potential negative consequences of limited HL without taxing the existing clinical workflow.
期刊介绍:
The Journal of Evaluation in Clinical Practice aims to promote the evaluation and development of clinical practice across medicine, nursing and the allied health professions. All aspects of health services research and public health policy analysis and debate are of interest to the Journal whether studied from a population-based or individual patient-centred perspective. Of particular interest to the Journal are submissions on all aspects of clinical effectiveness and efficiency including evidence-based medicine, clinical practice guidelines, clinical decision making, clinical services organisation, implementation and delivery, health economic evaluation, health process and outcome measurement and new or improved methods (conceptual and statistical) for systematic inquiry into clinical practice. Papers may take a classical quantitative or qualitative approach to investigation (or may utilise both techniques) or may take the form of learned essays, structured/systematic reviews and critiques.