Background: Artificial intelligence (AI) health care chatbots are gaining widespread adoption worldwide. It is imperative to understand the service quality of AI health care chatbots. However, there is limited guidance on how to comprehensively evaluate their service quality.
Objective: This study aimed to develop an index system based on the SERVQUAL framework for evaluating the service quality of AI health care chatbots.
Methods: An initial indicator pool was compiled through a comprehensive literature review and consultations with 4 experts. These indicators were mapped and categorized into 5 domains adapted from the SERVQUAL framework. The experts were recruited from hospital, university, and health commission settings by purposive sampling. The service quality index system was identified using a 2-round Delphi process, which included a virtual meeting between the 2 rounds. In the third round, indicator weights within each quality domain and subdomain were determined using the analytic hierarchy process.
Results: There were 26 indicators identified in the literature, based on which the 2-round Delphi process was conducted. A total of 20 experts were invited. The response rates in both rounds of Delphi and the analytic hierarchy process were 100%, and the authoritative coefficients were both >0.7. The final service quality index system for AI health care chatbots comprises 5 primary indicators and 17 secondary indicators. There were 3 (18%) indicators on assurance, 4 (24%) on reliability, 3 (18%) on human-likeness, 4 (24%) on tangibility, and 3 (18%) on responsiveness. The primary indicators, ranked from highest to lowest weight, were assurance (0.239), reliability (0.237), human-likeness (0.187), tangibility (0.170), and responsiveness (0.167).
Conclusions: This study pioneers the development of a service quality index system for AI health care chatbots adapted from the SERVQUAL framework. The results provide a validated tool for evaluating the performance of chatbots and offer valuable insights for health service managers and developers to enhance AI-driven medical consultation services.
Background: Medication adherence is vital for older adults living with type 2 diabetes mellitus (T2DM), but it remains low and needs improvement. Current interventions have limited effectiveness, while video-based interventions show promising potential for enhancing adherence.
Objective: To evaluate the impact of the "Diabetes Little Helper" video intervention, developed based on the information-motivation-behavioral skills model, on improving medication adherence in older adults living with T2DM in Henan.
Methods: This parallel-group randomized controlled trial was conducted in 2 hospitals in Zhengzhou, involving 68 patients from each hospital. The intervention group (IG) received standard care plus the video intervention for one month, while the control group (CG) received only standard care. The primary outcome was medication adherence, and secondary outcomes included medication knowledge, attitude, behavior, belief, and social support. Data were collected at baseline, postintervention, and at 3-month follow-up. Intention-to-treat analysis and the last observation carried forward method were applied for missing data, with the generalized estimating equation model used for effect assessment (P<.05 considered statistically significant).
Results: The average age of participants in the IG was 67.5 (SD 8.0) years, while in the CG, the average age was 66.0 (SD 7.0) years. Sex distribution was nearly identical, with 51.5% (n=35) of participants in the IG and 50.0% (n=34) in the CG being male. After the intervention, the IG and CG had retention rates of 95.6% (n=65) and 97.1% (n=66), respectively. At the 3-month follow-up, the retention rates for the IG and CG were 92.6% (n=63) and 92.2% (n=62), respectively. Both postintervention (β=4.956, 95% CI 3.702-6.210, P<.001) and at the 3-month follow-up (β=3.691, 95% CI 2.379-5.003, P<.001), medication adherence score for the IG was significantly higher than that for the CG. In addition, the IG showed significant improvements in secondary outcome, with medication knowledge (β=11.592, 95% CI 6.923-16.260, P<.001), attitude (β=5.467, 95% CI 4.531-6.763, P<.001), behavior (β=4.176, 95% CI 3.220-5.133, P<.001), and belief (β=2.882, 95% CI 1.990-3.775, P<.001) compared with the CG postintervention. However, there was no statistically significant difference in the secondary outcome of social support (β=0.008, 95% CI -1.834 to 2.011, P=.928).
Conclusions: The Diabetes Little Helper video intervention effectively improved medication adherence in older adults living with T2DM in Henan, highlighting the potential of digital health tools for managing chronic diseases in older adult populations.
Trial registration: Chinese Clinical Trial Registry ChiCTR2400083282; https://www.chictr.org.cn/showprojEN.html?proj=214847.

