NPJ Digital Medicine最新文献

Urinary tract infections (UTIs) often prompt empiric outpatient antibiotic prescriptions, risking mismatches. This study evaluates the impact of “UTI Smart-Set” (UTIS), an AI-driven decision-support tool, on prescribing patterns and mismatches in a large outpatient organization. UTIS integrates machine learning forecasts of antibiotic resistance, patient data, and guidelines into a user-friendly order set for UTI management. From 6/1/2021–8/31/2022, 171,010 UTI diagnoses were recorded, with UTIS used in 75,630 cases involving antibiotic prescriptions. Overall acceptance rate of UTIS recommendations was 66.0%. Among 19,287 cases with urine cultures, antibiotic mismatch rate was significantly lower when UTIS recommendations were followed (8.9% vs. 14.2%, p < 0.0001). Among women over 18, mismatch rate was 47.5% lower, and among women over 50, 55.6% lower (p < 0.001). Additionally, an overall reduction of 80.5% in ciprofloxacin usage (6.4% vs 32.9%, p < 0.0001) was observed. UTIS improved prescribing accuracy, reduced mismatches, and minimized quinolone use, highlighting AI’s potential for personalized infection management.

Predictive model performance may deteriorate when applied to data sources that were not used for training, thus, external validation is a key step in successful model deployment. As access to patient-level external data sources is typically limited, we recently proposed a method that estimates external model performance using only external summary statistics. Here, we benchmark the proposed method on multiple tasks using five large heterogeneous US data sources, where each, in turn, plays the role of an internal source and the remaining—external. Results showed accurate estimations for all metrics: 95th error percentiles for the area under the receiver operating characteristics (discrimination), calibration-in-the-large (calibration), Brier and scaled Brier scores (overall accuracy) of 0.03, 0.08, 0.0002, and 0.07, respectively. These results demonstrate the feasibility of estimating the transportability of prediction models using an internal cohort and external statistics. It may become an important accelerator of model deployment.

The use of synthetic data is a promising solution to facilitate the sharing and reuse of health-related data beyond its initial collection while addressing privacy concerns. However, there is still no consensus on a standardized approach for systematically evaluating the privacy and utility of synthetic data, impeding its broader adoption. In this work, we present a comprehensive review and systematization of current methods for evaluating synthetic health-related data, focusing on both privacy and utility aspects. Our findings suggest that there are a variety of methods for assessing the utility of synthetic data, but no consensus on which method is optimal in which scenario. Moreover, we found that most studies included in this review do not evaluate the privacy protection provided by synthetic data, and those that do often significantly underestimate the risks.

Chatbot-based multimodal AI holds promise for collecting medical histories and diagnosing ophthalmic diseases using textual and imaging data. This study developed and evaluated the ChatGPT-powered Intelligent Ophthalmic Multimodal Interactive Diagnostic System (IOMIDS) to enable patient self-diagnosis and self-triage. IOMIDS included a text model and three multimodal models (text + slit-lamp, text + smartphone, text + slit-lamp + smartphone). The performance was evaluated through a two-stage cross-sectional study across three medical centers involving 10 subspecialties and 50 diseases. Using 15640 data entries, IOMIDS actively collected and analyzed medical history alongside slit-lamp and/or smartphone images. The text + smartphone model showed the highest diagnostic accuracy (internal: 79.6%, external: 81.1%), while other multimodal models underperformed or matched the text model (internal: 69.6%, external: 72.5%). Moreover, triage accuracy was consistent across models. Multimodal approaches enhanced response quality and reduced misinformation. This proof-of-concept study highlights the potential of chatbot-based multimodal AI for self-diagnosis and self-triage. (The clinical trial was registered on June 26, 2023, on ClinicalTrials.gov under the registration number NCT05930444.).

In this study, we present MedS-Bench, a comprehensive benchmark to evaluate large language models (LLMs) in clinical contexts, MedS-Bench, spanning 11 high-level clinical tasks. We evaluate nine leading LLMs, e.g., MEDITRON, Llama 3, Mistral, GPT-4, Claude-3.5, etc. and found that most models struggle with these complex tasks. To address these limitations, we developed MedS-Ins, a large-scale instruction-tuning dataset for medicine. MedS-Ins comprises 58 medically oriented language corpora, totaling 5M instances with 19K instructions, across 122 tasks. To demonstrate the dataset’s utility, we conducted a proof-of-concept experiment by performing instruction tuning on a lightweight, open-source medical language model. The resulting model, MMedIns-Llama 3, significantly outperformed existing models on various clinical tasks. To promote further advancements, we have made MedS-Ins fully accessible and invite the research community to contribute to its expansion. Additionally, we have launched a dynamic leaderboard for MedS-Bench, to track the development progress of medical LLMs.

Early screening and evaluation of infant motor development are crucial for detecting motor deficits and enabling timely interventions. Traditional clinical assessments are often subjective, without fully capturing infants’ “real-world” behavior. This has sparked interest in portable, low-cost technologies to objectively and precisely measure infant motion at home, with a goal of enhancing ecological validity. In this systematic review, we explored the current landscape of portable, technology-based solutions to assess early motor development (within the first year), outlining the prevailing challenges and future directions. We reviewed 66 publications, which utilized video, sensors, or a combination of technologies. There were three key applications of these technologies: (1) automating clinical assessments, (2) illuminating new measures of motor development, and (3) predicting developmental outcomes. There was a promising trend toward earlier and more accurate detection using portable technologies. Additional research and demographic diversity are needed to develop fully automated, robust, and user-friendly tools. Registration & Protocol OSF Registries https://doi.org/10.17605/OSF.IO/R6JAE.

There is increasing use of digital tools to monitor people with psychosis and schizophrenia remotely, but using this type of data is challenging. This systematic review aimed to summarise how studies processed and analysed data collected through digital devices. In total, 203 articles collecting passive data through smartphones or wearable devices, from participants with psychosis or schizophrenia were included in the review. Accelerometers were the most common device (n = 115 studies), followed by smartphones (n = 46). The most commonly derived features were sleep duration (n = 50) and time spent sedentary (n = 41). Thirty studies assessed data quality and another 69 applied data quantity thresholds. Mixed effects models were used in 21 studies and time-series and machine-learning methods were used in 18 studies. Reporting of methods to process and analyse data was inconsistent, highlighting a need to improve the standardisation of methods and reporting in this area of research.

Deep-learning models have shown promise in differentiating between benign and malignant lesions. Previous studies have primarily focused on specific anatomical regions, overlooking tumors occurring throughout the body with highly heterogeneous whole-body backgrounds. Using neurofibromatosis type 1 (NF1) as an example, this study developed highly accurate MRI-based deep-learning models for the early automated screening of malignant peripheral nerve sheath tumors (MPNSTs) against complex whole-body background. In a Chinese seven-center cohort, data from 347 subjects were analyzed. Our one-step model incorporated normal tissue/organ labels to provide contextual information, offering a solution for tumors with complex backgrounds. To address privacy concerns, we utilized a lightweight deep neural network suitable for hospital deployment. The final model achieved an accuracy of 85.71% for MPNST diagnosis in the validation cohort and 84.75% accuracy in the independent test set, outperforming another classic two-step model. This success suggests potential for AI models in screening other whole-body primary/metastatic tumors.

Noninvasive methods for intracranial pressure (ICP) monitoring have emerged, but none has successfully replaced invasive techniques. This observational study developed and tested a machine learning (ML) model to estimate ICP using waveforms from a cranial extensometer device (brain4care [B4C] System). The model explored multiple waveform parameters to optimize mean ICP estimation. Data from 112 neurocritical patients with acute brain injuries were used, with 92 patients randomly assigned to training and testing, and 20 reserved for independent validation. The ML model achieved a mean absolute error of 3.00 mmHg, with a 95% confidence interval within ±7.5 mmHg. Approximately 72% of estimates from the validation sample were within 0-4 mmHg of invasive ICP values. This proof-of-concept study demonstrates that noninvasive ICP estimation via the B4C System and ML is feasible. Prospective studies are needed to validate the model’s clinical utility across diverse settings.

Polysomnography (PSG) is crucial for diagnosing sleep disorders, but manual scoring of PSG is time-consuming and subjective, leading to high variability. While machine-learning models have improved PSG scoring, their clinical use is hindered by the ‘black-box’ nature. In this study, we present SleepXViT, an automatic sleep staging system using Vision Transformer (ViT) that provides intuitive, consistent explanations by mimicking human ‘visual scoring’. Tested on KISS–a PSG image dataset from 7745 patients across four hospitals–SleepXViT achieved a Macro F1 score of 81.94%, outperforming baseline models and showing robust performances on public datasets SHHS1 and SHHS2. Furthermore, SleepXViT offers well-calibrated confidence scores, enabling expert review for low-confidence predictions, alongside high-resolution heatmaps highlighting essential features and relevance scores for adjacent epochs’ influence on sleep stage predictions. Together, these explanations reinforce the scoring consistency of SleepXViT, making it both reliable and interpretable, thereby facilitating the synergy between the AI model and human scorers in clinical settings.