Journal of Medical Internet Research最新文献

The COVID-19-Curated and Open Analysis and Research Platform (CO-CONNECT) project worked with 22 organizations across the United Kingdom to build a federated platform, enabling researchers to instantaneously and dynamically query federated datasets to find relevant data for their study. Finding relevant data takes time and effort, reducing the efficiency of research. Although data controllers could understand the value of such a system, there were significant challenges and delays in setting up the platform in response to COVID-19. This paper aims to present the challenges and lessons learned from the CO-CONNECT project to support other similar initiatives in the future. The project encountered many challenges, including the impacts of lockdowns on collaboration, understanding the new architecture, competing demands on people's time during a pandemic, data governance approvals, different levels of technical capabilities, data transformation to a common data model, access to granular-level laboratory data, and how to engage public and patient representatives meaningfully on a highly technical project. To overcome these challenges, we developed a range of methods to support data partners such as explainer videos; regular, short, "touch base" videoconference calls; drop-in workshops; live demos; and a standardized technical onboarding documentation pack. A 4-stage data governance process emerged. The patient and public representatives were fully integrated team members. Persistence, patience, and understanding were key. We make 8 recommendations to change the landscape for future similar initiatives. The new architecture and processes developed are being built upon for non-COVID-19-related data, providing an infrastructural legacy.

Background: Health information technologies, including electronic health records (EHRs), have revolutionized health care delivery. These technologies promise to enhance the efficiency and quality of care through improved patient health information management. Despite the transformative potential of EHRs, the extent to which patient access contributes to increased engagement with health care services within different clinical setting remains a distinct and underexplored facet.

Objective: This systematic review aims to investigate the impact of patient access to EHRs on health care engagement. Specifically, we seek to determine whether providing patients with access to their EHRs contributes to improved engagement with health care services.

Methods: A comprehensive systematic review search was conducted across various international databases, including Ovid MEDLINE, Embase, PsycINFO, and CINAHL, to identify relevant studies published from January 1, 2010, to November 15, 2023. The search on these databases was conducted using a combination of keywords and Medical Subject Heading terms related to patient access to electronic health records, patient engagement, and health care services. Studies were included if they assessed the impact of patient access to EHRs on health care engagement and provided evidence (quantitative or qualitative) for that. The guidelines of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 statement were followed for study selection, data extraction, and quality assessment. The included studies were assessed for quality using the Mixed Methods Appraisal Tool, and the results were reported using a narrative synthesis.

Results: The initial search from the databases yielded 1737 studies, to which, after scanning their reference lists, we added 10 studies. Of these 1747 studies, 18 (1.03%) met the inclusion criteria for the final review. The synthesized evidence from these studies revealed a positive relationship between patient access to EHRs and health care engagement, addressing 6 categories of health care engagement dimensions and outcomes, including treatment adherence and self-management, patient involvement and empowerment, health care communication and relationship, patient satisfaction and health outcomes, use of health care resources, and usability concerns and barriers.

Conclusions: The findings suggested a positive association between patient access to EHRs and health care engagement. The implications of these findings for health care providers, policy makers, and patients should be considered, highlighting the potential benefits and challenges associated with implementing and promoting patient access to EHRs. Further research directions have been proposed to deepen our understanding of this dynamic relationship.

Background: Social determinants of health (SDoH) such as housing insecurity are known to be intricately linked to patients' health status. More efficient methods for abstracting structured data on SDoH can help accelerate the inclusion of exposome variables in biomedical research and support health care systems in identifying patients who could benefit from proactive outreach. Large language models (LLMs) developed from Generative Pre-trained Transformers (GPTs) have shown potential for performing complex abstraction tasks on unstructured clinical notes.

Objective: Here, we assess the performance of GPTs on identifying temporal aspects of housing insecurity and compare results between both original and deidentified notes.

Methods: We compared the ability of GPT-3.5 and GPT-4 to identify instances of both current and past housing instability, as well as general housing status, from 25,217 notes from 795 pregnant women. Results were compared with manual abstraction, a named entity recognition model, and regular expressions.

Results: Compared with GPT-3.5 and the named entity recognition model, GPT-4 had the highest performance and had a much higher recall (0.924) than human abstractors (0.702) in identifying patients experiencing current or past housing instability, although precision was lower (0.850) compared with human abstractors (0.971). GPT-4's precision improved slightly (0.936 original, 0.939 deidentified) on deidentified versions of the same notes, while recall dropped (0.781 original, 0.704 deidentified).

Conclusions: This work demonstrates that while manual abstraction is likely to yield slightly more accurate results overall, LLMs can provide a scalable, cost-effective solution with the advantage of greater recall. This could support semiautomated abstraction, but given the potential risk for harm, human review would be essential before using results for any patient engagement or care decisions. Furthermore, recall was lower when notes were deidentified prior to LLM abstraction.

Background: Cognitive biases in clinical decision-making significantly contribute to errors in diagnosis and suboptimal patient outcomes. Addressing these biases presents a formidable challenge in the medical field.

Objective: This study aimed to explore the role of large language models (LLMs) in mitigating these biases through the use of the multi-agent framework. We simulate the clinical decision-making processes through multi-agent conversation and evaluate its efficacy in improving diagnostic accuracy compared with humans.

Methods: A total of 16 published and unpublished case reports where cognitive biases have resulted in misdiagnoses were identified from the literature. In the multi-agent framework, we leveraged GPT-4 (OpenAI) to facilitate interactions among different simulated agents to replicate clinical team dynamics. Each agent was assigned a distinct role: (1) making the final diagnosis after considering the discussions, (2) acting as a devil's advocate to correct confirmation and anchoring biases, (3) serving as a field expert in the required medical subspecialty, (4) facilitating discussions to mitigate premature closure bias, and (5) recording and summarizing findings. We tested varying combinations of these agents within the framework to determine which configuration yielded the highest rate of correct final diagnoses. Each scenario was repeated 5 times for consistency. The accuracy of the initial diagnoses and the final differential diagnoses were evaluated, and comparisons with human-generated answers were made using the Fisher exact test.

Results: A total of 240 responses were evaluated (3 different multi-agent frameworks). The initial diagnosis had an accuracy of 0% (0/80). However, following multi-agent discussions, the accuracy for the top 2 differential diagnoses increased to 76% (61/80) for the best-performing multi-agent framework (Framework 4-C). This was significantly higher compared with the accuracy achieved by human evaluators (odds ratio 3.49; P=.002).

Conclusions: The multi-agent framework demonstrated an ability to re-evaluate and correct misconceptions, even in scenarios with misleading initial investigations. In addition, the LLM-driven, multi-agent conversation framework shows promise in enhancing diagnostic accuracy in diagnostically challenging medical scenarios.

Background: The massive increase in the number of published scientific articles enhances knowledge but makes it more complicated to summarize results. The Medical Subject Headings (MeSH) thesaurus was created in the mid-20th century with the aim of systematizing article indexing and facilitating their retrieval. Despite the advent of search engines, few studies have questioned the relevance of the MeSH thesaurus, and none have done so systematically.

Objective: The objective of this study was to estimate the added value of using MeSH terms in PubMed queries for systematic reviews (SRs).

Methods: SRs published in 4 high-impact medical journals in general medicine over the past 10 years were selected. Only SRs for which a PubMed query was provided were included. Each query was transformed to obtain 3 versions: the original query (V1), the query with free-text terms only (V2), and the query with MeSH terms only (V3). These 3 queries were compared with each other based on their sensitivity and positive predictive values.

Results: In total, 59 SRs were included. The suppression of MeSH terms had an impact on the number of relevant articles retrieved for 24 (41%) out of 59 SRs. The median (IQR) sensitivities of queries V1 and V2 were 77.8% (62.1%-95.2%) and 71.4% (42.6%-90%), respectively. V1 queries provided an average of 2.62 additional relevant papers per SR compared with V2 queries. However, an additional 820.29 papers had to be screened. The cost of screening an additional collected paper was therefore 313.09, which was slightly more than triple the mean reading cost associated with V2 queries (88.67).

Conclusions: Our results revealed that removing MeSH terms from a query decreases sensitivity while slightly increasing the positive predictive value. Queries containing both MeSH and free-text terms yielded more relevant articles but required screening many additional papers. Despite this additional workload, MeSH terms remain indispensable for SRs.

Background: Current blood glucose monitoring (BGM) methods are often invasive and require repetitive pricking of a finger to obtain blood samples, predisposing individuals to pain, discomfort, and infection. Noninvasive blood glucose monitoring (NIBGM) is ideal for minimizing discomfort, reducing the risk of infection, and increasing convenience.

Objective: This review aimed to map the use cases of artificial intelligence (AI) in NIBGM.

Methods: A systematic scoping review was conducted according to the Arksey O'Malley five-step framework. Eight electronic databases (CINAHL, Embase, PubMed, Web of Science, Scopus, The Cochrane-Central Library, ACM Digital Library, and IEEE Xplore) were searched from inception until February 8, 2023. Study selection was conducted by 2 independent reviewers, descriptive analysis was conducted, and findings were presented narratively. Study characteristics (author, country, type of publication, study design, population characteristics, mean age, types of noninvasive techniques used, and application, as well as characteristics of the BGM systems) were extracted independently and cross-checked by 2 investigators. Methodological quality appraisal was conducted using the Checklist for assessment of medical AI.

Results: A total of 33 papers were included, representing studies from Asia, the United States, Europe, the Middle East, and Africa published between 2005 and 2023. Most studies used optical techniques (n=19, 58%) to estimate blood glucose levels (n=27, 82%). Others used electrochemical sensors (n=4), imaging (n=2), mixed techniques (n=2), and tissue impedance (n=1). Accuracy ranged from 35.56% to 94.23% and Clarke error grid (A+B) ranged from 86.91% to 100%. The most popular machine learning algorithm used was random forest (n=10) and the most popular deep learning model was the artificial neural network (n=6). The mean overall checklist for assessment of medical AI score on the included papers was 33.5 (SD 3.09), suggesting an average of medium quality. The studies reviewed demonstrate that some AI techniques can accurately predict glucose levels from noninvasive sources while enhancing comfort and ease of use for patients. However, the overall range of accuracy was wide due to the heterogeneity of models and input data.

Conclusions: Efforts are needed to standardize and regulate the use of AI technologies in BGM, as well as develop consensus guidelines and protocols to ensure the quality and safety of AI-assisted monitoring systems. The use of AI for NIBGM is a promising area of research that has the potential to revolutionize diabetes management.