Journal of the American College of Radiology : JACR最新文献

Objective: The purpose of this study is to examine the performance of Chat Generative Pre-trained Transformer (GPT)-4vision (GPT-4v) and GPT-4omni (GPT-4o) on the ACR's Diagnostic Radiology in-Training (DXIT) examination, comparing performance on image-based and text-only questions.

Methods: In all, 1,136 publicly available DXIT examination questions were input into GPT-4v and GPT-4o with a prompt asking the large language model to provide its answer, rationale, and confidence level (0-100). Accuracy of each model across different categories was then analyzed, with χ² tests to compare proportions, t tests to compare means, and receiver operating characteristic curves to evaluate confidence levels.

Results: GPT-4o and GPT-4v achieved accuracies of 73.5% and 69.3%, respectively (P < .0001) while scoring 55.6% and 50.3% on image-based questions (P < .0001). Receiver operating characteristic curves of confidence levels and correctness produced areas under the curve of 0.64 and 0.66 for GPT-4o and GPT-4v, respectively.

Discussion: GPT-4o outperformed GPT-4v on nearly every metric, with both models outperforming the national average performance of postgraduate year 3 radiology residents (61.9%) on the 2022 DXIT examination. However, performance on image-based questions remains significantly worse than text-only questions, and both models score below radiology trainees from the same cohort. Both models exhibit limited ability to predict correctness using an intrinsic confidence level. Use of ChatGPT for test preparation and image interpretation must therefore be approached with caution.

Purpose: To apply a Quality and Safety Continuous Process Improvement approach guided by Continuous Quality Improvement and Plan-Do-Study-Act (PDSA) cycles to develop, refine, and assess a digital reminder program's effect on Screening Mammography Missed Care Opportunity (SM-MCO) rates.

Methods: Study conducted at two Federally Qualified Community Health Centers and a mobile mammography unit. The pre-PDSA period was October 2020 to June 2023, and the post-PDSA period was July 2023 to January 2025. PDSA 1 launched a multilingual Short Messaging System (SMS) reminder across all sites. PDSA 2 standardized reminder process. PDSA 3 implemented a SM educational video. The primary outcome assessed the PDSA cycles' effect on SM-MCO rates. The secondary outcome assessed digital engagement. Quality improvement Statistical Process Control p-chart tracked appointment-level data. Univariate and logistic regression analyses assessed primary and secondary outcomes.

Results: In all, 18,654 appointments were included in the analysis; average age was 56.8 (SD = 9.6 years), and 51.9% identified as Hispanic. The overall SM-MCO rate declined from 29.2% pre-PDSA to 26.9% post-PDSA (P < .001). Appointments with SMS had a 35% SM-MCO rate, compared with 21.7% without (P < .001). Appointments with digital engagement had an SM-MCO rate of 21.7% compared with 40.4% without engagement (P < .001). Appointments that received and viewed the video had an SM-MCO rate of 11.5% compared with 26.9% without it (P < .001).

Conclusion: Although a modest decrease in overall SM-MCOs rate was observed, SM-MCO rates were higher among appointments that received SMS reminders but lower among appointments with digital engagement, underscoring the digital divide complexity. Quality Improvement frameworks can continuously monitor and refine digital strategies to increase access to radiology.

Objective: In the 26 years since its establishment, the Holman Research Pathway (HRP) has changed significantly. For example, a study published in 2018 found that interest among diagnostic radiology (DR) residents in the program had waned significantly, raising questions about the program's future. In this study, we sought to better understand the effectiveness of the HRP among DR residents, with a focus on the residency research productivity and career outcomes of DR residents who have completed the program.

Methods: We identified DR graduates of the HRP between 2003 and 2023 using the ABR's website and collected data regarding demographics, research output, and career outcomes from publicly available online sources. Research productivity was measured by first-author publications during residency and first- or last-author publications within 30 months after graduating from residency. Journal impact factors, citations, grant support, and open-access status were recorded. National Institutes of Health funding and academic employment were also evaluated.

Results: Thirty-three DR residents completed the HRP from 2003 to 2023 (mean 1.6 per year); 91% of graduates have completed subspecialty fellowships, 67% currently hold academic positions, and 27% have received National Institutes of Health funding. During training, residents published 64 first-author articles (mean 1.9 per resident) in journals with a median impact factor of 4.7, and 67% of these articles were published in open-access journals. In the first 30 months postresidency, graduates published a mean of 1.5 first- and last-author manuscripts in journals with a median impact factor of 3.5. There was a positive correlation between residency and postresidency research productivity (r = 0.5, P < .01).

Discussion: Although HRP participants in DR demonstrate research productivity comparable to radiation oncology graduates, fewer remain in academic positions, and overall participation has remained low. Increased awareness and support for the HRP may help attract more DR residents.

Artificial intelligence applications for radiology workflow have the potential to improve patient- and health system-level outcomes through more efficient and accurate diagnosis and clinical decision making. For a variety of time-intensive steps, numerous types of applications are now available with variable reported measures and degrees of success. The tools we highlight aim to accelerate imaging acquisition, reduce cognitive and manual burden on radiologists and others involved in the care pathway, improve diagnostic accuracy, and shorten the time to clinical action based on imaging results. Most existing studies have focused on intermediate outcomes, such as task duration or time to the next step in care. In this article, we present an examination of artificial intelligence applications across the medical imaging examination workflow, review examples of real-world evidence on these tools, and summarize the relevant performance metrics by application type. Beyond the more immediately acquired measures, to demonstrate benefit to patient health and economic outcomes, a more integrated assessment is necessary, and in an iterative fashion. To evolve beyond early workflow gains, interoperable tools must be tied to measurable downstream impacts, such as reduced disease severity, lower mortality, and shorter hospital stays, although we acknowledge that current empirical evaluations are limited.

Actionable findings requiring follow-up with additional imaging or other diagnostic procedures are frequently reported for a wide variety of radiology examinations. Completion of recommended follow-up can lead to new diagnoses including cancer. However, recommended follow-up completion is inconsistent, particularly when follow-up is for findings unrelated to the initial reason for the examination. Follow-up recommendation tracking systems, using a combination of information technology tools and human navigators, can facilitate completion of recommended follow-up, but often require significant effort for manual chart review and direct communication with providers and patients. Artificial intelligence, including large language models able to process vast and diverse unstructured text data, offer the opportunity to improve efficiency with data extraction and aggregation tasks, like those required for follow-up recommendation management. In this review article, we will review the key components of follow-up recommendation management systems: (1) identification of follow-up recommendations within radiology reports, (2) communication of these recommendations, (3) tracking of follow-up recommendations to completion, and (4) outcomes tracking. For each component, we will explore how artificial intelligence can improve efficiency and expand capabilities of robust management systems that ensure the loop is closed for follow-up recommendations.