JAMIA Open最新文献

Objectives: This study aims to investigate whether different types of electronic health record (EHR) users have distinct preferences for data quality assessment indicators (DQAI) and explore how these preferences can guide the enhancement of EHR systems and the optimization of related policies.

Materials and methods: High-frequency indicators were identified by a systematic literature review to construct a DQAI system, which was assessed by a user-oriented investigation involving doctors, nurses, hospital supervisors, and clinical researchers. The entropy weight method and fuzzy comprehensive evaluation model were employed for the system comprehensive evaluation. Exploratory factor analysis was used to construct dimensions, and visualization analysis was utilized to explore preferences at both the indicator and dimension levels.

Results: Sixteen indicators were identified to construct the DQAI system and grouped into 2 dimensions: structural and relational. The DQAI system achieved a comprehensive evaluation score of 90.445, corresponding to a "very important" membership level (62.5%). Doctors and nurses exhibited a higher score mean (4.43-4.66 out of 5) than supervisors (3.73-4.55 out of 5). Researchers emphasized credibility, with a score mean of 4.79 out of 5.

Discussion: The findings reveal that different types of EHR users exhibit distinct preferences for the DQAI at both indicator and dimension levels. Doctors and nurses thought that all indicators were important, clinical researchers emphasized credibility, and supervisors focused mainly on accuracy. Indicators in the relational dimension were generally more valued than structural ones. Doctors and nurses prioritized indicators of relational dimension, while researchers and supervisors leaned towards indicators of structural dimension. These insights suggest that tailored approaches in EHR system development and policy-making could enhance EHR data quality.

Conclusion: This study underscores the importance of user-centered approaches in optimizing EHR systems, highlighting diverse user preferences at both indicator and dimension levels.

Objectives: As artificial intelligence evolves, integrating speech processing into home healthcare (HHC) workflows is increasingly feasible. Audio-recorded communications enhance risk identification models, with automatic speech recognition (ASR) systems as a key component. This study evaluates the transcription accuracy and equity of 4 ASR systems-Amazon Web Services (AWS) General, AWS Medical, Whisper, and Wave2Vec-in transcribing patient-nurse communication in US HHC, focusing on their ability in accurate transcription of speech from Black and White English-speaking patients.

Materials and methods: We analyzed audio recordings of patient-nurse encounters from 35 patients (16 Black and 19 White) in a New York City-based HHC service. Overall, 860 utterances were available for study, including 475 drawn from Black patients and 385 from White patients. Automatic speech recognition performance was measured using word error rate (WER), benchmarked against a manual gold standard. Disparities were assessed by comparing ASR performance across racial groups using the linguistic inquiry and word count (LIWC) tool, focusing on 10 linguistic dimensions, as well as specific speech elements including repetition, filler words, and proper nouns (medical and nonmedical terms).

Results: The average age of participants was 67.8 years (SD = 14.4). Communication lasted an average of 15 minutes (range: 11-21 minutes) with a median of 1186 words per patient. Of 860 total utterances, 475 were from Black patients and 385 from White patients. Amazon Web Services General had the highest accuracy, with a median WER of 39%. However, all systems showed reduced accuracy for Black patients, with significant discrepancies in LIWC dimensions such as "Affect," "Social," and "Drives." Amazon Web Services Medical performed best for medical terms, though all systems have difficulties with filler words, repetition, and nonmedical terms, with AWS General showing the lowest error rates at 65%, 64%, and 53%, respectively.

Discussion: While AWS systems demonstrated superior accuracy, significant disparities by race highlight the need for more diverse training datasets and improved dialect sensitivity. Addressing these disparities is critical for ensuring equitable ASR performance in HHC settings and enhancing risk prediction models through audio-recorded communication.

Objective: The quality of alcohol-related prevention and treatment in US primary care is poor. The purpose of this study was to describe the extent to which Electronic Health Records (EHRs) used by 167 primary care practices across 7 states currently include the necessary prompts, clinical support, and performance reporting essential for improving alcohol-related prevention and treatment in primary care.

Materials and methods: Experts from five regional quality improvement programs identified basic EHR features needed to support evidence-based alcohol-related prevention (ie, screening and brief intervention) and treatment of alcohol use disorders (AUD). Data were collected regarding whether EHRs included these features.

Results: EHRs from 21 vendors were used by the primary care practices. For prevention, 62% of the 167 practices' EHRs included a validated screening questionnaire, 46% automatically scored the screening instrument, 62% could report the percent screened, and 37% could report the percent screening positive. Only 7% could report the percent offered brief intervention. For alcohol treatment, 49% of practices could report the percent diagnosed with AUD, 58% and 91% allowed documentation of referral and treatment with AUD medication, respectively. Only 3% could report the percent of patients diagnosed with AUD who received treatment.

Discussion: Most EHRs observed across 167 primary care practices across 7 US states lacked basic functionality necessary to support evidence-based alcohol-related prevention and AUD treatment. Only 3% and 7% of EHRs, respectively, included the ability to report widely recommended quality measures needed to improve the quality of recommended alcohol-related prevention and treatment in primary care.

Conclusion: Improving EHR functionality is likely necessary before alcohol-related primary care can be improved.

Objectives: Using machine learning models to predict infants at risk of defaulting routine immunization (RI) and identify significant features for Uganda.

Materials and methods: Principal component analysis reduced dimensionality. Datasets were balanced using synthetic minority over-sampling technique. k-Nearest Neighbors, Decision Trees, Random Forests (RFs), Support Vector Machine (SVM), Naïve-Bayes, Logistic Regression (LR), XGBoost, Adoptive-Boosting, and Gradient-Boosting were used on Uganda's 2016 Demographic and health survey data with social-economic and demographic factors as predictors. Experiments with and without K-fold cross-validation were performed. Models were evaluated for accuracy, recall, precision, and area under a curve (AUC).

Results and discussion: Experimental results revealed that the rate of defaulting increases as an infant's age increases at 5.3% Bacille Calmette-Guérin (BCG), 7.3% pentavalentI, 22.9% pentavalentIII, and 22.1% for measles. Significant predictors for BCG were immunization card, polio0, cluster altitude. Reception of pneumococcal1, BCG, and district for pentavalentI; polio3, pentavalentII for pentavalentIII; polio active and pentavalentIII for measles.RF had the best performance at predicting vaccine defaulting with 96%, 95%, 94%, 84% accuracy for BCG, PentavalentI, pentavalentIII, measles, respectively. Similarly, RF had the same precision, recall, AUC at 1.0. However, XGBoost, SVM, LR displayed the worst discriminatory power among infants who received the vaccine from defaulters with AUC ≤0.57.

Conclusion: Immunization card, preceding vaccines reception, and district were the most influential predictors. RF was the best classifier among the 9 models to predict defaulting RI. The study recommends regular outreaches, daily vaccination, provision of immunization cards, and accessible water sources to reduce defaulting.

Objective: Population health management (PHM) is a promising data-driven approach to address the challenges faced by health care systems worldwide. Although Business Intelligence (BI) systems are known to be relevant for a data-driven approach, the usage for PHM is limited in its elaboration. To explore available scientific publications, a systematic review guided by PRISMA was conducted of mature BI initiatives to investigate their decision contexts and BI capabilities.

Materials and methods: PubMed, Embase, and Web of Science were searched for articles published from January 2012 through November 2023. Articles were included if they described a (potential) BI system for PHM goals. Additional relevant publications were identified through snowballing. Technological Readiness Levels were evaluated to select mature initiatives from the 29 initiatives found. From the 11 most mature systems the decision context (eg, patient identification, risk stratification) and BI capabilities (eg, data warehouse, linked biobank) were extracted.

Results: The initiatives found are highly fragmented in decision context and BI capabilities. Varied terminology is used and much information is missing. Impact on population's health is currently limited for most initiatives. Care Link, CommunityRx, and Gesundes Kinzigtal currently stand out in aligning BI capabilities with their decision contexts.

Discussion and conclusion: PHM is a data-driven approach that requires a coherent data strategy and understanding of decision contexts and user needs. Effective BI capabilities depend on this understanding. Designing public-private partnerships to protect intellectual property while enabling rapid knowledge development is crucial. Development of a framework is proposed for systematic knowledge building.

Objectives: Artificial intelligence (AI) proceeds through an iterative and evaluative process of development, use, and refinement which may be characterized as a lifecycle. Within this context, stakeholders can vary in their interests and perceptions of the ethical issues associated with this rapidly evolving technology in ways that can fail to identify and avert adverse outcomes. Identifying issues throughout the AI lifecycle in a systematic manner can facilitate better-informed ethical deliberation.

Materials and methods: We analyzed existing lifecycles from within the current literature for ethical issues of AI in healthcare to identify themes, which we relied upon to create a lifecycle that consolidates these themes into a more comprehensive lifecycle. We then considered the potential benefits and harms of AI through this lifecycle to identify ethical questions that can arise at each step and to identify where conflicts and errors could arise in ethical analysis. We illustrated the approach in 3 case studies that highlight how different ethical dilemmas arise at different points in the lifecycle.

Results discussion and conclusion: Through case studies, we show how a systematic lifecycle-informed approach to the ethical analysis of AI enables mapping of the effects of AI onto different steps to guide deliberations on benefits and harms. The lifecycle-informed approach has broad applicability to different stakeholders and can facilitate communication on ethical issues for patients, healthcare professionals, research participants, and other stakeholders.

Objectives: Conducting simulation testing with end-users is essential for facilitating successful implementation of new health information technologies. This study designed a standardized simulation testing process with a system prototype prior to implementation to help study teams identify the system's interpretability and feasibility from the end-user perspective and to effectively integrate new innovations into real-world clinical settings and workflows.

Materials and methods: A clinical simulation model was developed to test a new Clinical Decision Support (CDS) system outside of the clinical environment while maintaining high fidelity. A web-based CDS prototype, the "CONCERN Smart Application," which leverages clinical data to measure and express a patient's risk of deterioration on a 3-level scale ("low," "moderate," or "high"), and audiovisual-integrated materials, were used to lead simulation sessions.

Results: A total of 6 simulation sessions with 17 nurses were held to investigate how nurses interact with the CONCERN Smart application and how it influences their critical thinking, and clinical responses. Four themes were extracted from the simulation debriefing sessions and used to inform implementation strategies. The strategies include how the CDS should be improved for practical real-world use.

Discussion and conclusions: Standardized simulation testing procedures identified and informed the necessary CDS improvements, the enhancements needed for real-world use, and the training requirements to effectively prepare end-users for system go-live.

Objectives: We propose and validate a domain knowledge-driven classification model for diagnosing post-acute sequelae of SARS-CoV-2 infection (PASC), also known as Long COVID, using Electronic Health Records (EHRs) data.

Materials and methods: We developed a robust model that incorporates features strongly indicative of PASC or associated with the severity of COVID-19 symptoms as identified in our literature review. The XGBoost tree-based architecture was chosen for its ability to handle class-imbalanced data and its potential for high interpretability. Using the training data provided by the Long COVID Computation Challenge (L3C), which was a sample of the National COVID Cohort Collaborative (N3C), our models were fine-tuned and calibrated to optimize Area Under the Receiver Operating characteristic curve (AUROC) and the F1 score, following best practices for the class-imbalanced N3C data.

Results: Our age-stratified classification model demonstrated strong performance with an average 5-fold cross-validated AUROC of 0.844 and F1 score of 0.539 across the young adult, mid-aged, and older-aged populations in the training data. In an independent testing dataset, which was made available after the challenge was over, we achieved an overall AUROC score of 0.814 and F1 score of 0.545.

Discussion: The results demonstrated the utility of knowledge-driven feature engineering in a sparse EHR data and demographic stratification in model development to diagnose a complex and heterogeneously presenting condition like PASC. The model's architecture, mirroring natural clinician decision-making processes, contributed to its robustness and interpretability, which are crucial for clinical translatability. Further, the model's generalizability was evaluated over a new cross-sectional data as provided in the later stages of the L3C challenge.

Conclusion: The study proposed and validated the effectiveness of age-stratified, tree-based classification models to diagnose PASC. Our approach highlights the potential of machine learning in addressing the diagnostic challenges posed by the heterogeneity of Long-COVID symptoms.

Objectives: To enable interactive visualization of the vaginal microbiome across the pregnancy and facilitate discovery of novel insights and generation of new hypotheses.

Material and methods: Vaginal Microbiome Atlas during Pregnancy (VMAP) was created with R shiny to generate visualizations of structured vaginal microbiome data from multiple studies.

Results: VMAP (http://vmapapp.org) visualizes 3880 vaginal microbiome samples of 1402 pregnant individuals from 11 studies, aggregated via open-source tool MaLiAmPi. Visualized features include diversity measures, VALENCIA community state types, and composition (phylotypes, taxonomy) that can be filtered by various categories.

Discussion: This work represents one of the largest and most geographically diverse aggregations of the vaginal microbiome in pregnancy to date and serves as a user-friendly resource to further analyze vaginal microbiome data and better understand pregnancies and associated outcomes.

Conclusion: VMAP can be obtained from https://github.com/msirota/vmap.git and is currently deployed as an online app for non-R users.

Objectives: Development of search queries for systematic reviews (SRs) is time-consuming. In this work, we capitalize on recent advances in large language models (LLMs) and a relatively large dataset of natural language descriptions of reviews and corresponding Boolean searches to generate Boolean search queries from SR titles and key questions.

Materials and methods: We curated a training dataset of 10 346 SR search queries registered in PROSPERO. We used this dataset to fine-tune a set of models to generate search queries based on Mistral-Instruct-7b. We evaluated the models quantitatively using an evaluation dataset of 57 SRs and qualitatively through semi-structured interviews with 8 experienced medical librarians.

Results: The model-generated search queries had median sensitivity of 85% (interquartile range [IQR] 40%-100%) and number needed to read of 1206 citations (IQR 205-5810). The interviews suggested that the models lack both the necessary sensitivity and precision to be used without scrutiny but could be useful for topic scoping or as initial queries to be refined.

Discussion: Future research should focus on improving the dataset with more high-quality search queries, assessing whether fine-tuning the model on other fields, such as the population and intervention, improves performance, and exploring the addition of interactivity to the interface.

Conclusions: The datasets developed for this project can be used to train and evaluate LLMs that map review descriptions to Boolean search queries. The models cannot replace thoughtful search query design but may be useful in providing suggestions for key words and the framework for the query.