Healthcare Informatics Research最新文献

Objectives: Developing large language models (LLMs) in biomedicine requires access to high-quality training and alignment tuning datasets. However, publicly available Korean medical preference datasets are scarce, hindering the advancement of Korean medical LLMs. This study constructs and evaluates the efficacy of the Korean Medical Preference Dataset (KoMeP), an alignment tuning dataset constructed with an automated pipeline, minimizing the high costs of human annotation.

Methods: KoMeP was generated using the DAHL score, an automated hallucination evaluation metric. Five LLMs (Dolly-v2-3B, MPT-7B, GPT-4o, Qwen-2-7B, Llama-3-8B) produced responses to 8,573 biomedical examination questions, from which 5,551 preference pairs were extracted. Each pair consisted of a "chosen" response and a "rejected" response, as determined by their DAHL scores. The dataset was evaluated when trained through two different alignment tuning methods, direct preference optimization (DPO) and odds ratio preference optimization (ORPO) respectively across five different models. The KorMedMCQA benchmark was employed to assess the effectiveness of alignment tuning.

Results: Models trained with DPO consistently improved KorMedMCQA performance; notably, Llama-3.1-8B showed a 43.96% increase. In contrast, ORPO training produced inconsistent results. Additionally, English-to-Korean transfer learning proved effective, particularly for English-centric models like Gemma-2, whereas Korean-to-English transfer learning achieved limited success. Instruction tuning with KoMeP yielded mixed outcomes, which suggests challenges in dataset formatting.

Conclusions: KoMeP is the first publicly available Korean medical preference dataset and significantly improves alignment tuning performance in LLMs. The DPO method outperforms ORPO in alignment tuning. Future work should focus on expanding KoMeP, developing a Korean-native dataset, and refining alignment tuning methods to produce safer and more reliable Korean medical LLMs.

Objectives: This study proposes a mobile-based explainable artificial intelligence (XAI) platform designed for diagnosing febrile illnesses.

Methods: We integrated the interpretability offered by local interpretable model-agnostic explanations (LIME) and the explainability provided by generative pre-trained transformers (GPT) to bridge the gap in understanding and trust often created by machine learning models in critical healthcare decision-making. The developed system employed random forest for disease diagnosis, LIME for interpretation of the results, and GPT-3.5 for generating explanations in easy-to-understand language.

Results: Our model demonstrated robust performance in detecting malaria, achieving precision, recall, and F1-scores of 85%, 91%, and 88%, respectively. It performed moderately well in detecting urinary tract and respiratory tract infections, with precision, recall, and F1-scores of 80%, 65%, and 72%, and 77%, 68%, and 72%, respectively, maintaining an effective balance between sensitivity and specificity. However, the model exhibited limitations in detecting typhoid fever and human immunodeficiency virus/acquired immune deficiency syndrome, achieving lower precision, recall, and F1-scores of 69%, 53%, and 60%, and 75%, 39%, and 51%, respectively. These results indicate missed true-positive cases, necessitating further model fine-tuning. LIME and GPT-3.5 were integrated to enhance transparency and provide natural language explanations, thereby aiding decision-making and improving user comprehension of the diagnoses.

Conclusions: The LIME plots revealed key symptoms influencing the diagnoses, with bitter taste in the mouth and fever showing the highest negative influence on predictions, and GPT-3.5 provided natural language explanations that increased the reliability and trustworthiness of the system, promoting improved patient outcomes and reducing the healthcare burden.

Objectives: This research aimed to develop a retrieval-augmented generation (RAG) based large language model (LLM) system that offers personalized and reliable responses to a wide range of concerns raised by Korean adolescents. Our work focuses on building a culturally reflective dataset and on designing and validating the system's effectiveness by comparing the answer quality of RAG-based models with non-RAG models.

Methods: Data were collected from the NAVER Knowledge iN platform, concentrating on posts that featured adolescents' questions and corresponding expert responses during the period 2014-2024. The dataset comprises 3,874 cases, categorized by key negative emotions and the primary sources of worry. The data were processed to remove irrelevant or redundant content and then classified into general and detailed causes. The RAG-based model employed FAISS for similarity-based retrieval of the top three reference cases and used GPT-4o mini for response generation. The responses generated with and without RAG were evaluated using several metrics.

Results: RAG-based responses outperformed non-RAG responses across all evaluation metrics. Key findings indicate that RAG-based responses delivered more specific, empathetic, and actionable guidance, particularly when addressing complex emotional and situational concerns. The analysis revealed that family relationships, peer interactions, and academic stress are significant factors affecting adolescents' worries, with depression and stress frequently co-occurring.

Conclusions: This study demonstrates the potential of RAG-based LLMs to address the diverse and culture-specific worries of Korean adolescents. By integrating external knowledge and offering personalized support, the proposed system provides a scalable approach to enhancing mental health interventions for adolescents. Future research should concentrate on expanding the dataset and improving multiturn conversational capabilities to deliver even more comprehensive support.

Objectives: To identify the right interventions for the right heart failure (HF) patients in the real-world setting using machine learning (ML) trained on individual-level clinical data linked with social determinants of health (SDOH) data.

Methods: In this retrospective cohort study, point-of-care claims data from Komodo Health and SDOH data from the National Health and Wellness Survey (NHWS), from January 2014-December 2020, were linked. Data mining was conducted using K-means clustering, an ML tool. Komodo Health data were used to access longitudinal data for the selected patient cohorts and crosssectional data from NHWS for additional patient information. The primary outcome was HF-related hospitalizations; secondary outcomes, all-cause hospitalization and all-cause mortality. Use of digital healthcare (DHC)/non-DHC interventions and related outcomes were also assessed.

Results: The study population included 353 HF patients (mean age, 63.5 years; 57.2% women). The use of non-DHC (75.9%-81.9%) and DHC (4.0%-9.1%) interventions increased from baseline to followup. Overall, 17.0% of patients had HF-related hospitalizations (DHC, 6.9%; non-DHC, 16.5%) and 45.0% had all-cause hospitalization (DHC, 75.0%; non-DHC, 50.9%). Two archetypes with distinct patient profiles were identified. Archetype 1 (vs. 2) characterised by older age, greater disease severity, more comorbidities, more medication use, took steps to prevent heart attack/problems, had better lifestyle, higher HF-related hospitalizations (18.3% vs. 16.3%) and lower all-cause hospitalizations (42.9% vs. 46.3%). The trends remained the same regardless of the intervention type.

Conclusions: Identification of patient archetypes with distinct profiles can be useful to understand underlying disease subtypes, identify specific interventions, predict clinical outcomes, and define the right intervention for the right patient.

Objectives: We aimed to derive observational research evidence on treatment patterns through a scoping review of common data model (CDM)-based publications.

Methods: We searched the medical literature databases PubMed and EMBASE, as well as the Observational Health Data Sciences and Informatics (OHDSI) website, for papers published between January 1, 2010 and August 21, 2023 to identify research papers relevant to our topic.

Results: Eighteen articles satisfied the inclusion criteria for this scoping review. We summarized study characteristics such as phenotypes, patient numbers, data periods, countries, Observational Medical Outcomes Partnership (OMOP) CDM databases, and definitions of index date and target cohort. Type 2 diabetes mellitus emerged as the most frequently studied disease, covered in five articles, followed by hypertension and depression, each addressed in four articles. Biguanides, with metformin as the primary drug, were the most commonly prescribed first-line treatments for type 2 diabetes mellitus. Most studies utilized sunburst plots to visualize treatment patterns, whereas two studies used Sankey plots. Various software tools were employed for treatment pattern analysis, including JavaScript, the open-source ATLAS by OHDSI, R code, and the R package "TreatmentPatterns."

Conclusions: This study provides a comprehensive overview of research on treatment patterns using the CDM, highlighting the growing importance of OMOP CDM in enabling multinational observational network studies and advancing collaborative research in this field.

Objectives: A growing number of health professional institutions around the world are embracing innovative technologies to increase student engagement, primarily to improve clinical and simulated learning experiences. Didactic learning is an essential component of dental and medical curricula. However, limited research is available regarding the implementation of technology-infused teaching in classroom settings. We developed self-paced interactive learning content using the HTML5 Package (H5P) to promote student engagement in a didactic course within a dental hygiene program.

Methods: A total of 52 interactive artifacts were created and administered to students as supplementary learning material. A descriptive study was conducted to explore student perceptions and engagement with the H5P content, as well as to evaluate the impact of these artifacts on academic performance.

Results: Students performed significantly better on exam questions associated with interactive H5P content posted in the learning management system compared to other questions. Most students were highly engaged with the H5P content during the week leading up to each summative assessment. However, two of the three students with the highest course grades demonstrated consistent engagement with this content throughout the course.

Conclusions: Our results highlight the effectiveness of interactive content created using the H5P platform in fostering student engagement. The development of self-paced interactive materials may benefit various aspects of didactic teaching, including both synchronous and asynchronous online learning.

Objectives: Conventional face-to-face nutrition counseling has played a crucial role in promoting healthy habits. However, the emergence of digital health technologies has introduced mobile app-based nutrition counseling as an effective alternative. This research aims to develop and evaluate the usability and effectiveness of the FIT4PEDON mobile nutrition counseling application in promoting healthy dietary behaviors and lifestyle modifications among childhood cancer survivors (CCS).

Methods: This study employed a mixed-methods approach, incorporating both survey and qualitative and quantitative analyses. A total of 33 health care professional experts participated. The reliability of the questionnaire was assessed using the Kuder-Richardson method, and its content validity was confirmed through expert evaluation. Usability testing was conducted with a validated questionnaire.

Results: The development process resulted in two applications: an Android mobile application and an admin web application. The findings indicated that a significant proportion of experts endorsed the app for dietary management. Statistical analysis showed significant differences between "yes" and "no" responses. However, no significant differences were found when comparing responses across different sex or age groups.

Conclusions: The FIT4PEDON application shows promise in supporting CCS to adopt healthier lifestyles. Nevertheless, the study underscores the necessity for further research, particularly focusing on specific age groups of experts with relevant experience, to achieve more conclusive results. Leveraging technology through mobile apps has the potential to improve the quality of survivorship care and foster sustained engagement in long-term care for pediatric cancer survivors.

Objectives: Hypertension, commonly known as high blood pressure, is a prevalent and serious condition affecting a significant portion of the adult population globally. It is a chronic medical issue that, if left unaddressed, can lead to severe health complications, including kidney problems, heart disease, and stroke. This study aims to develop a feature selection model using the XGBoost algorithm to identify specific single nucleotide polymorphisms (SNPs) as biomarkers for detecting hypertension risk.

Methods: We propose using the high dimensionality of genetic variations (i.e., SNPs) to build a classifier model for prediction. In this study, SNPs were used as markers for hypertension in patients. We utilized the OpenSNP dataset, which includes 19,697 SNPs from 2,052 samples. Extreme gradient boosting (XGBoost) is an ensemble machine learning method employed here for feature selection, which incrementally adjusts weights in a series of steps.

Results: The experimental results identified 292 SNPs that exhibited high performance, with an F1-score of 98.55%, precision of 98.73%, recall of 98.38%, and overall accuracy of 98%. This study provides compelling evidence that the XGBoost feature selection method outperforms other representative feature selection methods, such as genetic algorithms, analysis of variance, chi-square, and principal component analysis, in predicting hypertension risk, demonstrating its effectiveness.

Conclusions: We developed a model for predicting hypertension using the SNPs dataset. The high dimensionality of SNP data was effectively managed to identify significant features as biomarkers using the XGBoost feature selection method. The results indicate high performance in predicting the risk of hypertension.

Objectives: Emergency department (ED) overcrowding significantly impacts healthcare efficiency, safety, and resource management. Predictive models that utilize triage information can streamline the admission process. This review evaluates existing hospital admission prediction models that have been developed or validated using triage data for adult ED patients.

Methods: A systematic search of PubMed, Embase, CINAHL, Web of Science, and the Cochrane Library was conducted. Studies were selected if they developed or validated predictive models for hospital admission using triage data from adult ED patients. Data extraction adhered to the CHARMS (Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies), and the risk of bias was evaluated using PROBAST (Prediction model Risk of Bias Assessment Tool).

Results: Twenty studies met the inclusion criteria, employing logistic regression and machine learning techniques. Logistic regression was noted for its traditional use and clinical interpretability, whereas machine learning provided enhanced flexibility and potential for better predictive accuracy. Common predictors included patient demographics, triage category, vital signs, and mode of arrival. The area under the curve values for model performance ranged from 0.80 to 0.89, demonstrating strong discriminatory ability. However, external validation was limited, and there was variability in outcome definitions and model generalizability.

Conclusions: Predictive models based on triage data show promise in supporting ED operations by facilitating early predictions of hospital admissions, which could help decrease boarding times and enhance patient flow. Further research is necessary to validate these models in various settings to confirm their applicability and reliability.