Journal of Medical Internet Research最新文献

Background: Large language models (LLMs) demonstrated advanced performance in processing clinical information. However, commercially available LLMs lack specialized medical knowledge and remain susceptible to generating inaccurate information. Given the need for self-management in diabetes, patients commonly seek information online. We introduce the Retrieval-augmented Information System for Enhancement (RISE) framework and evaluate its performance in enhancing LLMs to provide accurate responses to diabetes-related inquiries.

Objective: This study aimed to evaluate the potential of the RISE framework, an information retrieval and augmentation tool, to improve the LLM's performance to accurately and safely respond to diabetes-related inquiries.

Methods: The RISE, an innovative retrieval augmentation framework, comprises 4 steps: rewriting query, information retrieval, summarization, and execution. Using a set of 43 common diabetes-related questions, we evaluated 3 base LLMs (GPT-4, Anthropic Claude 2, Google Bard) and their RISE-enhanced versions respectively. Assessments were conducted by clinicians for accuracy and comprehensiveness and by patients for understandability.

Results: The integration of RISE significantly improved the accuracy and comprehensiveness of responses from all 3 base LLMs. On average, the percentage of accurate responses increased by 12% (15/129) with RISE. Specifically, the rates of accurate responses increased by 7% (3/43) for GPT-4, 19% (8/43) for Claude 2, and 9% (4/43) for Google Bard. The framework also enhanced response comprehensiveness, with mean scores improving by 0.44 (SD 0.10). Understandability was also enhanced by 0.19 (SD 0.13) on average. Data collection was conducted from September 30, 2023 to February 5, 2024.

Conclusions: The RISE significantly improves LLMs' performance in responding to diabetes-related inquiries, enhancing accuracy, comprehensiveness, and understandability. These improvements have crucial implications for RISE's future role in patient education and chronic illness self-management, which contributes to relieving medical resource pressures and raising public awareness of medical knowledge.

Background: Internet hospitals (IHs) have rapidly developed as a promising strategy to address supply-demand imbalances in China's medical industry, with their capabilities directly dependent on information platform functionality. Furthermore, a novel theory of "Trinity" smart hospital has provided advanced guidelines on IH constructions.

Objective: This study aimed to explore the construction experience, construction models, and development prospects based on operational data from IHs.

Methods: Based on existing information systems and internet service functionalities, our hospital has built a "Smart Hospital Internet Information Platform (SHIIP)" for IH operations, actively to expand online services, digitalize traditional health care, and explore health care services modes throughout the entire process and lifecycle. This article encompasses the platform architecture design, technological applications, patient service content and processes, health care professional support features, administrative management tools, and associated operational data.

Results: Our platform has presented a set of data, including 82,279,669 visits, 420,120 online medical consultations, 124,422 electronic prescriptions, 92,285 medication deliveries, 6,965,566 prediagnosis triages, 4,995,824 offline outpatient appointments, 2025 medical education articles with a total of 15,148,310 views, and so on. These data demonstrate the significant role of IH as an indispensable component of our physical hospital services, with deep integration between online and offline health care systems.

Conclusions: The upward trends in various data metrics indicate that our IH has gained significant recognition and usage among both the public and healthcare workers, and may have promising development prospects. Additionally, the platform construction approach, which prioritizes comprehensive service digitization and the 'Trinity' of the public, healthcare workers, and managers, serves as an effective means of promoting the development of Internet Hospitals. Such insights may prove invaluable in guiding the development of IH and facilitating the continued evolution of the Internet healthcare sector.

Background: Internet hospitals, which refer to service platforms that integrate consultation, prescription, payment, and drug delivery based on hospital entities, have been developing at a rapid pace in China since 2014. However, assessments regarding their service quality and patient satisfaction have not been well developed. There is an urgent need to comprehensively evaluate and improve the service quality of internet hospitals.

Objective: This study aims to investigate the current status of patients' use of internet hospitals, as well as familiarity and willingness to use internet hospitals, to evaluate patients' expected and perceived service qualities of internet hospitals using the Chinese version of the Service Quality Questionnaire (SERVQUAL-C) with a national representative sample, and to explore the association between service quality of internet hospitals and patients' overall satisfaction toward associated medical platforms.

Methods: This cross-sectional survey was conducted through face-to-face or digital interviews from June to September 2022. A total of 1481 outpatient participants (635 men and 846 women; mean age 33.22, SD 13.22). Participants reported their use of internet hospitals, and then rated their expectations and perceptions of service quality toward internet hospitals via the SERVQUAL-C, along with their demographic information.

Results: Among the surveyed participants, 51.2% (n=758) of participants had used internet hospital service or services. Use varied across age, education level, and annual income. Although the majority of them (n=826, 55.8%) did not know internet hospital services well, 68.1% (n=1009) of participants expressed the willingness to adopt this service. Service quality evaluation revealed that the perceived service quality did not match with the expectation, especially the responsiveness dimension. Important-performance analysis results further alerted that reliable diagnosis, prompt response, clear feedback pathway, and active feedback handling were typically the services awaiting substantial improvement. More importantly, multiple linear regressions revealed that familiarity and willingness to use internet hospital services were significant predictors of satisfaction, above and over tangibles, reliability, and empathy service perspectives, and demographic characteristics such as gender, age, education level, and annual income.

Conclusions: In the future, internet hospitals should focus more on how to narrow the gaps between the expected and perceived service quality. Promotion of internet hospitals should also be facilitated to increase patients' familiarity with and willingness to use these services.