Healthcare Informatics Research最新文献

Objectives: Large language models (LLMs) are increasingly used in clinical practice, but their performance can deteriorate when radiology reports are incomplete. We evaluated whether multimodal LLMs (integrating text and images) could enhance accuracy and interpretability in chest radiography reports, thereby improving their utility for clinical decision support. Specifically, we aimed to assess the robustness of LLMs in generating accurate impressions from chest radiography reports when provided with incomplete data, and whether multimodal input could mitigate performance loss.

Methods: We analyzed 300 radiology image-report pairs from the MIMIC-CXR database. Three LLMs-OpenFlamingo, MedFlamingo, IDEFICS-were tested in text-only and multimodal formats. Chest X-ray impressions were generated from complete text reports and then regenerated after systematically removing 20%, 50%, and 80% of the text. The effect of adding images was evaluated using chest X-rays, and model performance was compared using three statistical methods. Hallucination rates were quantified.

Results: In the text-only setting, OpenFlamingo, MedFlamingo, and IDEFICS demonstrated comparable performance (ROUGE-L: 0.23 vs. 0.21 vs. 0.21; F1RadGraph: 0.20 vs. 0.16 vs. 0.16; F1CheXbert: 0.49 vs. 0.41 vs. 0.41), with OpenFlamingo performing best on complete text (p < 0.001). All models exhibited performance decline with incomplete data. However, multimodal input significantly improved the performance of MedFlamingo and IDEFICS (p < 0.001), equaling or surpassing OpenFlamingo even under incomplete text conditions. Regarding hallucination, MedFlamingo showed a lower false-negative rate in multimodal compared with unimodal use, while false-positive rates were similar.

Conclusions: LLMs may produce suboptimal outputs when radiology data are incomplete, but multimodal LLMs enhance reliability and may strengthen clinical decision-making support.

Objectives: Cardiovascular diseases have been the leading cause of death worldwide. The American Heart Association defined eight metrics for cardiovascular health to reduce mortality. Mobile health tools can support shared clinical decisionmaking, provide tele-monitoring feedback, and improve patient adherence to medication regimens. This work aims to develop and implement the Cardiovascular Health application for mobile phones according to the parameters defined by the American Heart Association.

Methods: A user-centered design approach was employed using the Dart programming language, the Flutter framework, and a Firebase database.

Results: Each ideal parameter is evaluated as "good" when it meets the requirements, earning the patient one mark. Participants' cardiovascular health is subsequently classified as "good," "can be improved," or "needs to be improved," and PDF reports are generated.

Conclusions: The Cardiovascular Health application is built on a strong scientific foundation, given the high prevalence of individuals at risk for cardiovascular disease. It includes all components necessary to assess cardiovascular health and will enable physicians and other healthcare professionals to make more informed decisions regarding patient care.

Objectives: The Korean government has been developing the National Integrated Biological Data Construction Project (NIBDCP) for over a decade, aiming to establish a comprehensive framework for the collection, production, provision, and utilization of biological data. This study examines the project's structure, features, and governance framework to identify key recommendations for successful implementation.

Methods: A systematic analysis of the NIBDCP was conducted, focusing on governance structures, data management protocols, and operational systems. The evaluation emphasized institutional roles, consent requirements, sustainable data production, and researcher accessibility, identifying areas for improvement.

Results: The analysis identified four critical areas requiring enhancement. First, the governance framework should empower the Secretariat to clearly define institutional responsibilities and facilitate inter-agency collaboration. Second, data collection protocols must address broad consent requirements, including provision of adequate information, explicit consent for secondary use, itemized withdrawal options, protection of minors' rights, and improved participant convenience. Third, establishing a systemic and sustainable data production framework is essential, with an emphasis on data quality, standardization, and scalability. Finally, the system for data provision and utilization should enhance researcher accessibility by ensuring data openness, maintaining a unified Institutional Review Board system, and streamlining application and usage processes.

Conclusions: Strengthening governance, upholding ethical standards in data collection, ensuring sustainable data production, and optimizing researcher accessibility are essential for the success of the NIBDCP. These measures will help achieve the project's goals and establish a robust model for biological data governance and utilization in Korea.

Objectives: This study was conducted to investigate public perceptions and concerns surrounding tuberculosis (TB) treatment in Korea through an analysis of online queries about antitubercular medications. Additionally, it evaluated the effectiveness of large language models (LLMs) as analytical tools for processing unstructured healthcare data.

Methods: Using LLMs, this study analyzed 44,174 questions that mentioned TB from Naver Knowledge-iN (2002-2024). Questions referencing antitubercular medications were extracted and thematically categorized. Side effects were analyzed through parallel approaches examining general and medication-specific effects. Questions about infectivity and social implications were further analyzed using text embedding, dimensionality reduction, and clustering. The performance of LLMs was evaluated against human researchers and traditional methods.

Results: Among questions mentioning specific medications (n = 919), rifampin (31.8%) and isoniazid (31.6%) were most frequently referenced. Of the 10,044 questions regarding antitubercular medication, management challenges represented the largest category (44.8%). Analysis of infectivity and social implications (n = 583) revealed previously unidentified concerns about blood donation and immigration eligibility. Employment-related concerns constituted the largest distinct subgroup (20.6%). Hepatotoxicity, dermatosis, and vomiting were the most frequently reported side effects. LLMs outperformed keyword matching in data processing and offered cost advantages over human analysis, with finetuning further reducing processing costs.

Conclusions: This study produced novel insights into public concerns regarding TB treatment and demonstrated the effectiveness of combining social media platform data with LLM-based analysis, providing a systematic framework for future healthcare research using unstructured public data and LLMs.

Objectives: The objective of this study was to evaluate the effectiveness of machine learning (ML) models using selected subsets of features to predict age based on intracranial arterial segments' tortuosity and diameter characteristics derived from magnetic resonance angiography (MRA) data. Additionally, this study aimed to identify key vascular features important for predicting vascular age.

Methods: Three-dimensional time-of-flight MRA image data from 171 subjects were analyzed. After annotating the endpoints for each arterial segment, 169 features-comprising tortuosity metrics and arterial segment diameter statistics-were extracted. Five ML models (random forest, linear regression, AdaBoost, XGBoost, and lightGBM) were trained and validated. Two feature selection methods, correlation-based feature selection (CFS) and Relief-F, were applied to identify optimal feature subsets.

Results: The random forest model utilizing the CFS-based 50% feature subset achieved the best performance, with a root mean square error of 14.0 years, a coefficient of determination (R2) of 0.275, and a Pearson correlation coefficient of 0.560. Tortuosity metrics (e.g., triangular index of the left posterior cerebral artery P1 segment) appeared more frequently than diameter statistics among the top five most important features.

Conclusions: CFS-based feature selection enhanced the performance of ML-based age prediction compared with using the complete feature set. Linear regression consistently demonstrated the poorest performance across all evaluation metrics. ML-based age prediction using segmental tortuosity metrics and diameter statistics is feasible, potentially revealing significant features related to vascular aging.

Objectives: This study aimed to develop and evaluate a retrieval-augmented generation (RAG)-based chatbot system designed to optimize hospital operations. By leveraging electronic medical record (EMR) manuals, the system seeks to streamline administrative workflows and enhance healthcare delivery.

Methods: The system integrated fine-tuned multilingual embedding models (Multilingual-E5-Large and BGE-M3) for indexing and retrieving information from EMR manuals. A dataset comprising 5,931 question-document pairs was constructed through query augmentation and validated by domain experts. Fine-tuning was performed using contrastive learning to enhance semantic understanding, with performance assessed using top-k accuracy metrics. The Solar Mini Chat API was adopted for text generation, prioritizing Korean-language responses and cost efficiency.

Results: The fine-tuned models demonstrated marked improvements in retrieval accuracy, with BGE-M3 achieving 97.6% and Multilingual-E5-Large reaching 89.7%. The chatbot achieved high performance, with query latency under 10 ms and robust retrieval precision, effectively addressing operational EMR queries. Key applications included administrative task support and billing process optimization, highlighting its potential to reduce staff workload and enhance healthcare service delivery.

Conclusions: The RAG-based chatbot system successfully addressed critical challenges in healthcare administration, improving EMR usability and operational efficiency. Future research should focus on realworld deployment and longitudinal studies to further evaluate its impact on administrative burden reduction and workflow improvement.

Objectives: This study proposes the integration of chatbots into personalized medicine by demonstrating how these tools can support the personalized medicine model. Chatbots can deliver tailored health recommendations, facilitate patient-doctor communication, and provide decision support in clinical settings. The goal is to establish a reference framework aligned with national and international standards for personalized healthcare solutions.

Methods: The chatbot model was developed by reviewing 30 scientific and academic articles focused on artificial intelligence and natural language processing in healthcare. The study analyzed the capabilities of existing healthcare chatbots, particularly their capacity to support personalized medicine through accurate data collection and processing of individual health information.

Results: Key parameters identified for effective chatbot deployment in personalized medicine include user engagement, data accuracy, adaptability, and regulatory compliance. The study established a compliance benchmark of 25% based on current industry standards and application performance. The results indicate that the proposed chatbot model significantly increased the precision and efficacy of personalized medical recommendations, surpassing baseline requirements set by standardization organizations.

Conclusions: This model provides healthcare professionals and patients with a robust framework for utilizing chatbots in personalized medicine, focusing on improved patient outcomes and engagement. The research identifies a gap in the application of artificial intelligence-driven tools in personalized healthcare and suggests strategic directions for future innovations. Implementing this model aims to bridge this gap, offering a standardized approach to developing chatbots that support personalized medicine.

Objectives: This study aims to predict the severity of Parkinson's disease (PD) by leveraging a comprehensive dataset integrating cerebrospinal fluid protein and peptide data sourced from UniProt, normalized protein expression metrics, clinical assessments, and gait data. The dataset comprised 248 PD patients monitored longitudinally, with periodic evaluations including 227 proteins, 971 peptides, gait parameters, and Movement Disorder Society-sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS) scores at baseline 0, 6, 12, and 24 months.

Methods: A multifaceted machine learning framework was employed, consisting of random forest, TensorFlow decision forests, and a custom-developed phaseshift ensembling model. Additionally, regression techniques such as linear regression, random forest regressor, decision tree regressor, and K-nearest neighbors were utilized to support the predictions. These models aimed to forecast PD severity as reflected by UPDRS scores.

Results: The custom phase-shift ensembling model demonstrated superior predictive performance, achieving an average symmetric mean absolute percentage error (sMAPE) of 55 across all UPDRS sections. Notably, the random forest regressor excelled in predicting motor function severity (UPDRS-III), attaining an sMAPE of 77.32, indicating its ability to model complex disease progression dynamics effectively.

Conclusions: Integrating biological markers, clinical scores, and gait dynamics facilitates accurate modeling of PD progression. The ensemble-based approach, particularly phase-shift ensembling, improves prediction robustness and interpretability, offering a powerful strategy for the early prediction of PD severity. This study highlights the value of multi-source data fusion and advanced machine learning techniques in supporting early diagnosis and informed treatment planning for neurodegenerative diseases.

Objectives: This study assessed the effectiveness of in-context learning using Generative Pre-trained Transformer-4 (GPT-4) for labeling radiology reports.

Methods: In this retrospective study, radiology reports were obtained from the Medical Information Mart for Intensive Care III database. Two structured prompts-the "basic prompt" and the "in-context prompt"- were compared. An optimization experiment was conducted to assess consistency and the occurrence of output format errors. The primary labeling experiments were performed on 200 unseen head computed tomography (CT) reports for multilabel classification of predefined labels (Experiment 1) and on 400 unseen abdominal CT reports for multi-label classification of actionable findings (Experiment 2).

Results: The inter-reader accuracies in Experiments 1 and 2 were 0.93 and 0.84, respectively. For multi-label classification of head CT reports (Experiment 1), the in-context prompt led to notable increases in F1-scores for the "foreign body" and "mass" labels (gains of 0.66 and 0.22, respectively). However, improvements for other labels were modest. In multi-label classification of abdominal CT reports (Experiment 2), in-context prompts produced substantial improvements in F1-scores across all labels compared to basic prompts. Providing context equipped the model with domain-specific knowledge and helped align its existing knowledge, thereby improving performance.

Conclusions: Incontext learning with GPT-4 consistently improved performance in labeling radiology reports. This approach is particularly effective for subjective labeling tasks and allows the model to align its criteria with those of human annotators for objective labeling. This practical strategy offers a simple, adaptable, and researcher-oriented method that can be applied to diverse labeling tasks.