Applied Clinical Informatics最新文献

Background: Health informatics education is pivotal in integrating diversity, equity, inclusion, and accessibility (DEIA) principles into curricula and leveraging data with equity considerations. Integrating clinically driven data with other datasets is crucial to comprehensive understanding of patient care demographics, experiences, and outcomes to create equity-minded data storytelling. Publicly available Healthy People 2030 (HP2030) resources complement academic EHRs, supporting tailored learning activities in informatics education to enhance educational utility through a DEIA lens.

Objectives: This case report describes the expansion of an existing DEI checklist to an updated DEIA checklist for preparing future informaticians to collect and critically evaluate DEIA features using this checklist in creating equity-minded data storytelling.

Methods: An equity-minded data storytelling model and the HP2030 framework were utilized to develop the DEIA checklist. We employed an informal cognitive walkthrough to expand the DEIA checklist and evaluate the DEIA measures or characteristics within datasets from the HP2030 social determinants of health (SDOH) 5 topics using this checklist.

Results: We reviewed 76 available SDOH-related datasets and added 6 measures to "demographics" and 7 to "skills, abilities, & accessibility" of the DEIA checklist. Our evaluation of the DEIA checklist verified HP2030's inclusion of all measures, except "religions/beliefs." All DEIA measures were linked to equity and accessibility, 1 in inclusion, and the inclusion of 3 characteristics comprising the category "language" and 6 characteristics comprising the category "images."

Conclusion: Results highlighted the accessibility and comprehensiveness of HP2030 demographic data resources, considering SDOH factors and promoting inclusive data representation to address health disparities. The DEIA checklist provides a structured tool in facilitating unbiased data collection and visualization of SDOH-related data in data storytelling through an equity-informed lens. Integrating an equity-minded data storytelling with frameworks like HP2030 enriches health informatics education, broadens students' understanding of health disparities, and supports evidence-based interventions for improved health outcomes.

Objectives: The main objective of this study is to evaluate the ability of the Large Language Model ChatGPT to accurately answer USMLE board style medical ethics questions compared to medical knowledge based questions. This study has the additional objectives of comparing the overall accuracy of GPT-3.5 to GPT-4 and to assess the variability of responses given by each version.

Materials and methods: Using AMBOSS, a third party USMLE Step Exam test prep service, we selected one group of 27 medical ethics questions and a second group of 27 medical knowledge questions matched on question difficulty for medical students. We ran 30 trials asking these questions on GPT-3.5 and GPT-4, and recorded the output. A random-effects linear probability regression model evaluated accuracy, and a Shannon entropy calculation evaluated response variation.

Results: Both versions of ChatGPT demonstrated a worse performance on medical ethics questions compared to medical knowledge questions. GPT-4 performed 18% points (P < 0.05) worse on medical ethics questions compared to medical knowledge questions and GPT-3.5 performed 7% points (P = 0.41) worse. GPT-4 outperformed GPT-3.5 by 22% points (P < 0.001) on medical ethics and 33% points (P < 0.001) on medical knowledge. GPT-4 also exhibited an overall lower Shannon entropy for medical ethics and medical knowledge questions (0.21 and 0.11, respectively) than GPT-3.5 (0.59 and 0.55) which indicates lower variability in response.

Conclusion: Both versions of ChatGPT performed more poorly on medical ethics questions compared to medical knowledge questions. GPT-4 significantly outperformed GPT-3.5 on overall accuracy and exhibited a significantly lower response variability in answer choices. This underscores the need for ongoing assessment of ChatGPT versions for medical education.

Key words: ChatGPT, Large Language Model, Artificial Intelligence, Medical Education, USMLE, Ethics.

Background: Clinical Decision Support (CDS) tools have a mixed record of effectiveness, often due to inadequate alignment with clinical workflows and poor usability. While there's a consensus that usability testing methods address these issues, in practice, usability testing is generally only used for selected projects (such as funded research studies). There is a critical need for CDS operations to apply usability testing to all CDS implementations.

Objectives: In this State of the Art / Best Practice paper, we share challenges with scaling usability in healthcare operations and alternative methods and CDS governance structures to enable usability testing as a routine practice.

Methods: We coalesce our experience and results of applying guerilla in-situ usability testing to over 20 projects in 1 year period with the proposed solution.

Results: We demonstrate the feasibility of adopting "guerilla in-situ usability testing" in operations and their effectiveness in incorporating user feedback and improving design.

Conclusion: Although some methodological rigor was relaxed to accommodate operational speed, the benefits outweighed the limitations. Broader adoption of usability testing may transform CDS implementation and improve health outcomes.

Background:  Millions of Americans manage their health care with the help of a trusted individual. Shared access to a patient's online patient portal is one tool that can assist their care partner(s) in gaining access to the patient's health information and allow for easy exchange with the patient's care team. Shared access provides care partners with a validated and secure method for accessing the patient's portal account using their own login credentials. Shared access provides extra privacy protection and control to the patient, who designates which individuals can view their record. It also reduces confusion for the care team when interacting with the care partner via the portal. Shared access is underutilized among adult patients' care partners.

Objectives:  Investigate the process of granting or receiving shared access at multiple health care organizations in the United States to learn about barriers and facilitators experienced by patients and care partners.

Methods:  The Shared Access Learning Collaborative undertook a "Secret Shopper" exercise. Participants attempted to give or gain shared access to another adult's portal account. After each attempt they completed a 14-question survey with a mix of open- and closed-ended questions.

Results:  Eighteen participants attempted to grant or receive shared access a total of 24 times. Fifteen attempts were successful. Barriers to success included requiring paper forms with signatures, lack of knowledgeable staff, lack of access to technical support, and difficult-to-navigate technology. Facilitators included easy-to-navigate online processes and accessible technical help. Participants who were successful in gaining shared access reported feeling more informed and able to engage in shared decision-making.

Conclusion:  The outcomes of our secret shopper exercise underscore the importance of collaboration aimed at learning from diverse encounters and disseminating the best practices. This is essential to address technical, informational, and organizational obstacles that may impede the widespread and accessible adoption of shared access.

Background:  The Accreditation Council for Graduate Medical Education suggests that Clinical Informatics (CI) fellowship programs foster broad skills, which include collaboration and project management. However, they do not dictate how to best accomplish these learning objectives.

Objectives:  This study aimed to describe a standard approach to project-based learning for CI, to share its implementation, and to discuss lessons learned.

Methods:  We created a standard approach to project-based learning based on concepts from adult learning theory, the project life cycle framework, the Toyota Production System, and Improvement Science.

Results:  With this standard approach in place, we learned how best to support fellows in its use. In addition to this approach to supporting needs assessment, risk/change management, implementation, and evaluation/improvement skills, we found the need to develop fellow skills in collaboration, leadership, and time management/managing up. Supported by project-based learning using this standard approach, and with targeted project selection to meet topic-based learning objectives, fellows reached the ability to practice independently in 15 to 21 months.

Discussion:  Fellows are uniquely positioned to ensure the success of projects due to their increased availability and protected time compared with attendings. They are readily available for project teams to draw upon their expertise with clinical workflows and understanding of technological solutions. Project-based learning addressing organizational priorities complements fellow project management coursework and improves fellows' ability to function successfully in large, complex, and dynamic organizations. Exposing fellows to contemporary problems, then addressing them through projects, provides fellows with up-to-date applied informatics knowledge.

Conclusion:  Project-based learning can ensure that many general CI learning objectives are supported inherently. It reinforces project management teachings, while providing fellows with a marketable project portfolio to aid with future job applications. Having projects tightly aligned with organizational priorities supports ongoing investment in fellowship programs.

Background:  Over the past 30 years, the American Medical Informatics Association (AMIA) has played a pivotal role in fostering a collaborative community for professionals in biomedical and health informatics. As an interdisciplinary association, AMIA brings together individuals with clinical, research, and computer expertise and emphasizes the use of data to enhance biomedical research and clinical work. The need for a recognition program within AMIA, acknowledging applied informatics skills by members, led to the establishment of the Fellows of AMIA (FAMIA) Recognition Program in 2018.

Objectives:  To outline the evolution of the FAMIA program and shed light on its origins, development, and impact. This report explores factors that led to the establishment of FAMIA, considerations affecting its development, and the objectives FAMIA seeks to achieve within the broader context of AMIA.

Methods:  The development of FAMIA is examined through a historical lens, encompassing key milestones, discussions, and decisions that shaped the program. Insights into the formation of FAMIA were gathered through discussions within AMIA membership and leadership, including proposals, board-level discussions, and the involvement of key stakeholders. Additionally, the report outlines criteria for FAMIA eligibility and the pathways available for recognition, namely the Certification Pathway and the Long-Term Experience Pathway.

Results:  The FAMIA program has inducted five classes, totaling 602 fellows. An overview of disciplines, roles, and application pathways for FAMIA members is provided. A comparative analysis with other fellow recognition programs in related fields showcases the unique features and contributions of FAMIA in acknowledging applied informatics.

Conclusion:  Now in its sixth year, FAMIA acknowledges the growing influence of applied informatics within health information professionals, recognizing individuals with experience, training, and a commitment to the highest level of applied informatics and the science associated with it.

Background:  The method of documentation during a clinical encounter may affect the patient-physician relationship.

Objectives:  Evaluate how the use of ambient voice recognition, coupled with natural language processing and artificial intelligence (DAX), affects the patient-physician relationship.

Methods:  This was a prospective observational study with a primary aim of evaluating any difference in patient satisfaction on the Patient-Doctor Relationship Questionnaire-9 (PDRQ-9) scale between primary care encounters in which DAX was utilized for documentation as compared to another method. A single-arm open-label phase was also performed to query direct feedback from patients.

Results:  A total of 288 patients were include in the open-label arm and 304 patients were included in the masked phase of the study comparing encounters with and without DAX use. In the open-label phase, patients strongly agreed that the provider was more focused on them, spent less time typing, and made the encounter feel more personable. In the masked phase of the study, no difference was seen in the total PDRQ-9 score between patients whose encounters used DAX (median: 45, interquartile range [IQR]: 8) and those who did not (median: 45 [IQR: 3.5]; p = 0.31). The adjusted odds ratio for DAX use was 0.8 (95% confidence interval: 0.48-1.34) for the patient reporting complete satisfaction on how well their clinician listened to them during their encounter.

Conclusion:  Patients strongly agreed with the use of ambient voice recognition, coupled with natural language processing and artificial intelligence (DAX) for documentation in primary care. However, no difference was detected in the patient-physician relationship on the PDRQ-9 scale.

Background:  Clinical decision support systems (CDSSs) are computer applications, which can be applied to give guidance to practitioners in antimicrobial stewardship (AS) activities; however, further information is needed for their optimal use.

Objectives:  Our objective was to analyze the implementation of a CDSS program in a second-level hospital, describing alerts, recommendations, and the effects on consumption and clinical outcomes.

Methods:  In October 2020, a pharmacist-driven CDSS designed for AS was implemented in a second-level hospital. The program provides a list of alerts related to antimicrobial treatment and microbiology, which were automatized for revision by the AS professionals. To analyze the implementation of the CDSS, a pre-post-intervention, retrospective study was designed. AS-triggered alerts and recommendations (total number and rate of acceptance) were compiled. The effect of the CDSS was measured using antimicrobial consumption, duration of antimicrobial treatments, in-hospital mortality, and length of stay (LOS) for patients admitted for infectious causes.

Results:  The AS team revised a total of 7,543 alerts and 772 patients had at least one recommendation, with an acceptance rate of 79.3%. Antimicrobial consumption decreased from 691.1 to 656.8 defined daily doses (DDD)/1,000 beds-month (p = 0.04) and the duration of antimicrobial treatment from 3.6 to 3.3 days (p < 0.01). In-hospital mortality decreased from 6.6 to 6.2% (p = 0.46) and mean LOS from 7.2 to 6.2 days (p < 0.01).

Conclusion:  The implementation of a CDSS resulted in a significant reduction of antimicrobial DDD, duration of antimicrobial treatments, and hospital LOS. There was no significant difference in mortality.

Objective:  The overall goal of this work is to create a patient-reported outcome (PRO) and decision support system to help postpartum patients determine when to seek care for concerning symptoms. In this case study, we assessed differences in perspectives for application design needs based on race, ethnicity, and preferred language.

Methods:  A sample of 446 participants who reported giving birth in the past 12 months was recruited from an existing survey panel. We sampled participants from four self-reported demographic groups: (1) English-speaking panel, Black/African American race, non-Hispanic ethnicity; (2) Spanish-speaking panel, Hispanic-ethnicity; (3) English-speaking panel, Hispanic ethnicity; (4) English-speaking panel, non-Black race, non-Hispanic ethnicity. Participants provided survey-based feedback regarding interest in using the application, comfort reporting symptoms, desired frequency of reporting, reporting tool features, and preferred outreach pathway for concerning symptoms.

Results:  Fewer Black participants, compared with all other groups, stated that they had used an app for reporting symptoms (p = 0.02), were least interested in downloading the described application (p < 0.05), and found a feature for sharing warning sign information with friends and family least important (p < 0.01). Black and non-Hispanic Black participants also preferred reporting symptoms less frequently as compared with Hispanic participants (English and Spanish-speaking; all p < 0.05). Spanish-speaking Hispanic participants tended to prefer calling their professional regarding urgent warning signs, while Black and English-speaking Hispanic groups tended to express interest in using an online chat or patient portal (all p < 0.05) CONCLUSION:  Different participant groups described distinct preferences for postpartum symptom reporting based on race, ethnicity, and preferred languages. Tools used to elicit PROs should consider how to be flexible for different preferences or tailored toward different groups.

Background: Numerous programs have arisen to address interruptive clinical decision support (CDS) with the goals of reducing alert burden and alert fatigue. These programs often have standing committees with broad stakeholder representation, significant governance efforts, and substantial analyst hours to achieve reductions in alert burden which can be difficult for hospital systems to replicate.

Objective:  This study aimed to reduce nursing alert burden with a primary nurse informaticist and small support team through a quality-improvement approach focusing on high-volume alerts.

Methods:  Target alerts were identified from the period of January 2022 to April 2022 and four of the highest firing alerts were chosen initially, which accounted for 43% of all interruptive nursing alerts and an estimated 86 hours per month of time across all nurses occupied resolving these alerts per month. Work was done concurrently for each alert with design changes based on the Five Rights of CDS and following a quality-improvement framework. Priority for work was based on operational engagement for design review and approval. Once initial design changes were approved, alerts were taken for in situ usability testing and additional changes were made as needed. Final designs were presented to stakeholders for approval prior to implementation.

Results:  The total number of interruptive nursing alert firings decreased by 58% from preintervention period (1 January 2022-30 June 2022) to postintervention period (July 1, 2022-December 31, 2022). Action taken on alerts increased from 8.1 to 17.3%. The estimated time spent resolving interruptive alerts summed across all nurses in the system decreased from 197 hours/month to 114 hours/month.

Conclusion:  While CDS may improve use of evidence-based practices, implementation without a clear framework for evaluation and monitoring often results in alert burden and fatigue without clear benefits. An alert burden reduction effort spearheaded by a single empowered nurse informaticist efficiently reduced nursing alert burden substantially.