Jake G. Stenzel MS, Nicholas R. Schultz MS, Michael J. Marino MD
{"title":"耳鼻喉科医生在线评论的自动分类。","authors":"Jake G. Stenzel MS, Nicholas R. Schultz MS, Michael J. Marino MD","doi":"10.1002/lio2.70036","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Objectives</h3>\n \n <p>The study aimed to extract online comments of otolaryngologists in the 20 most populated cities in the United States from healthgrades.com, develop and validate a natural language processing (NLP) logistic regression algorithm for automated text classification of reviews into 10 categories, and compare 1- and 5-star reviews in directly-physician-related and non-physician-related categories.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>1977 1-star and 12,682 5-star reviews were collected. The primary investigator manually categorized a training dataset of 324 1-star and 909 5-star reviews, while a validation subset of 100 5-star and 50 1-star reviews underwent dual manual categorization. Using scikit-learn, an NLP algorithm was trained and validated on the subsets, with F1 scores evaluating text classification accuracy against manual categorization. The algorithm was then applied to the entire dataset with comparison of review categorization according to 1- and 5-star reviews.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>F1 scores for NLP validation ranged between 0.71 and 0.97. Significant associations emerged between 1-star reviews and treatment plan, accessibility, wait time, office scheduling, billing, and facilities. 5-star reviews were associated with surgery/procedure, bedside manner, and staff/mid-levels.</p>\n </section>\n \n <section>\n \n <h3> Conclusion</h3>\n \n <p>The study successfully validated an NLP text classification system for categorizing online physician reviews. Positive reviews were found to have an association with directly-physician related context. 1-star reviews were related to treatment plan, accessibility, wait time, office scheduling, billing, and facilities. This method of text classification effectively discerned the nuances of human-written text, providing valuable insights into online healthcare feedback that is scalable.</p>\n \n <p>Level of evidence: Level 3</p>\n </section>\n </div>","PeriodicalId":48529,"journal":{"name":"Laryngoscope Investigative Otolaryngology","volume":"9 6","pages":""},"PeriodicalIF":1.6000,"publicationDate":"2024-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11558699/pdf/","citationCount":"0","resultStr":"{\"title\":\"Automated classification of online reviews of otolaryngologists\",\"authors\":\"Jake G. Stenzel MS, Nicholas R. Schultz MS, Michael J. Marino MD\",\"doi\":\"10.1002/lio2.70036\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div>\\n \\n \\n <section>\\n \\n <h3> Objectives</h3>\\n \\n <p>The study aimed to extract online comments of otolaryngologists in the 20 most populated cities in the United States from healthgrades.com, develop and validate a natural language processing (NLP) logistic regression algorithm for automated text classification of reviews into 10 categories, and compare 1- and 5-star reviews in directly-physician-related and non-physician-related categories.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Methods</h3>\\n \\n <p>1977 1-star and 12,682 5-star reviews were collected. The primary investigator manually categorized a training dataset of 324 1-star and 909 5-star reviews, while a validation subset of 100 5-star and 50 1-star reviews underwent dual manual categorization. Using scikit-learn, an NLP algorithm was trained and validated on the subsets, with F1 scores evaluating text classification accuracy against manual categorization. The algorithm was then applied to the entire dataset with comparison of review categorization according to 1- and 5-star reviews.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Results</h3>\\n \\n <p>F1 scores for NLP validation ranged between 0.71 and 0.97. Significant associations emerged between 1-star reviews and treatment plan, accessibility, wait time, office scheduling, billing, and facilities. 5-star reviews were associated with surgery/procedure, bedside manner, and staff/mid-levels.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Conclusion</h3>\\n \\n <p>The study successfully validated an NLP text classification system for categorizing online physician reviews. Positive reviews were found to have an association with directly-physician related context. 1-star reviews were related to treatment plan, accessibility, wait time, office scheduling, billing, and facilities. This method of text classification effectively discerned the nuances of human-written text, providing valuable insights into online healthcare feedback that is scalable.</p>\\n \\n <p>Level of evidence: Level 3</p>\\n </section>\\n </div>\",\"PeriodicalId\":48529,\"journal\":{\"name\":\"Laryngoscope Investigative Otolaryngology\",\"volume\":\"9 6\",\"pages\":\"\"},\"PeriodicalIF\":1.6000,\"publicationDate\":\"2024-11-13\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11558699/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Laryngoscope Investigative Otolaryngology\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/lio2.70036\",\"RegionNum\":4,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"OTORHINOLARYNGOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Laryngoscope Investigative Otolaryngology","FirstCategoryId":"3","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/lio2.70036","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"OTORHINOLARYNGOLOGY","Score":null,"Total":0}
Automated classification of online reviews of otolaryngologists
Objectives
The study aimed to extract online comments of otolaryngologists in the 20 most populated cities in the United States from healthgrades.com, develop and validate a natural language processing (NLP) logistic regression algorithm for automated text classification of reviews into 10 categories, and compare 1- and 5-star reviews in directly-physician-related and non-physician-related categories.
Methods
1977 1-star and 12,682 5-star reviews were collected. The primary investigator manually categorized a training dataset of 324 1-star and 909 5-star reviews, while a validation subset of 100 5-star and 50 1-star reviews underwent dual manual categorization. Using scikit-learn, an NLP algorithm was trained and validated on the subsets, with F1 scores evaluating text classification accuracy against manual categorization. The algorithm was then applied to the entire dataset with comparison of review categorization according to 1- and 5-star reviews.
Results
F1 scores for NLP validation ranged between 0.71 and 0.97. Significant associations emerged between 1-star reviews and treatment plan, accessibility, wait time, office scheduling, billing, and facilities. 5-star reviews were associated with surgery/procedure, bedside manner, and staff/mid-levels.
Conclusion
The study successfully validated an NLP text classification system for categorizing online physician reviews. Positive reviews were found to have an association with directly-physician related context. 1-star reviews were related to treatment plan, accessibility, wait time, office scheduling, billing, and facilities. This method of text classification effectively discerned the nuances of human-written text, providing valuable insights into online healthcare feedback that is scalable.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
