Journal of the American College of Radiology : JACR最新文献

Objective: This study explores radiology program directors' perspectives on the impact of large language model (LLM) use among residency applicants to craft personal statements.

Methods: Eight program directors from the Radiology Residency Education Research Alliance participated in a mixed-methods study, which included a survey regarding impressions of artificial intelligence (AI)-generated personal statements and focus group discussions (July 2023). Each director reviewed four personal statement variations for five applicants, anonymized to author type: the original and three Chat Generative Pre-trained Transformer-4.0 (GPT) versions generated with varying prompts, aggregated for analysis. A 5-point Likert scale surveyed the writing quality, including voice, clarity, engagement, organization, and perceived origin of each statement. An experienced qualitative researcher facilitated focus group discussions. Data analysis was performed using a rapid analytic approach with a coding template capturing key areas related to residency applications.

Results: GPT-generated statement ratings were more often average or worse in quality (56%, 268 of 475) than ratings of human-authored statements (29%, 45 of 160). Although reviewers were not confident in their ability to distinguish the origin of personal statements, they did so reliably and consistently, identifying the human-authored personal statements at 95% (38 of 40) as probably or definitely original. Focus group discussions highlighted the inevitable use of AI in crafting personal statements and concerns about its impact on the authenticity and the value of the personal statement in residency selections. Program directors were divided on the appropriate use and regulation of AI.

Discussion: Radiology residency program directors rated LLM-generated personal statements as lower in quality and expressed concern about the loss of the applicant's voice but acknowledged the inevitability of increased AI use in the generation of application statements.

Purpose: To assess the ability of the Annalise Enterprise CXR Triage Trauma (Annalise AI Pty Ltd, Sydney, NSW, Australia) artificial intelligence model to identify vertebral compression fractures on chest radiographs and its potential to address undiagnosed osteoporosis and its treatment.

Materials and methods: This retrospective study used a consecutive cohort of 596 chest radiographs from four US hospitals between 2015 and 2021. Each radiograph included both frontal (anteroposterior or posteroanterior) and lateral projections. These radiographs were assessed for the presence of vertebral compression fracture in a consensus manner by up to three thoracic radiologists. The model then performed inference on the cases. A chart review was also performed for the presence of osteoporosis-related International Classification of Diseases, 10th revision diagnostic codes and medication use for the study period and an additional year of follow-up.

Results: The model successfully completed inference on 595 cases (99.8%); these cases included 272 positive cases and 323 negative cases. The model performed with area under the receiver operating characteristic curve of 0.955 (95% confidence interval [CI]: 0.939-0.968), sensitivity 89.3% (95% CI: 85.7%-92.7%) and specificity 89.2% (95% CI: 85.4%-92.3%). Out of the 236 true-positive cases (ie, correctly identified vertebral compression fractures by the model) with available chart information, only 86 (36.4%) had a diagnosis of vertebral compression fracture and 140 (59.3%) had a diagnosis of either osteoporosis or osteopenia; only 78 (33.1%) were receiving a disease-modifying medication for osteoporosis.

Conclusion: The model identified vertebral compression fracture accurately with a sensitivity 89.3% (95% CI: 85.7%-92.7%) and specificity of 89.2% (95% CI: 85.4%-92.3%). Its automated use could help identify patients who have undiagnosed osteoporosis and who may benefit from taking disease-modifying medications.

Objective: Despite the pervasiveness and adverse impacts of financial toxicity (FT) in cancer care, there are no definitive measures for FT screening that have been widely integrated into clinical practice. The aim of this review is to evaluate current methods of assessing FT among patients with cancer and confirm factors associated with higher risk of FT.

Methods: A systematic review was performed according to PRISMA guidelines. We included peer-reviewed studies that cross-sectionally, longitudinally, or prospectively measured the self-reported financial impact of patients undergoing cancer care in the United States.

Results: Out of 1,085 identified studies, 51 met final inclusion criteria. Outcomes evaluated included FT measures or tools, time and setting of screening, FT prevalence, and sociodemographic or clinical patient-level associated factors. Our findings demonstrate that there is wide variability in FT screening practices including in the timing (diagnosis versus treatment versus survivorship), setting (clinic-based, online, telephone or mail), tools used (21 unique tools, 7 previously validated), and interpretations of screening results (varying FT score cutoffs defining high versus low FT). Younger age, lower income, lower education, non-White race, employment status change, advanced cancer stage, and systemic or radiation therapy were among factors associated with worse FT across the studies.

Discussion: FT screening remains heterogenous within the United States. With the ever-escalating cost of cancer care, and the strong association between FT and poor patient outcomes, universal and routine FT screening is imperative in cancer care. Further research and multifaceted interventions identifying best practices for FT screening are needed.

Objective: Two-tiered preference signaling has been implemented in the radiology residency application system to reduce congestion in the setting of high-volume applications. Signals are an indicator of strong interest that an applicant can transmit to a limited number of programs. This study assessed the impact of program signaling on interview invitations, how applicants strategically used signals based on their application's competitiveness, and applicants' attitudes toward the current signaling system.

Methods: A survey was sent to radiology residency applicants registered with TheRadRoom during the 2024 application cycle. We queried the applicants' background, applications, signal distribution, and interview outcome depending on the type of signal sent. We also asked whether respondents received an interview invitation from a hypothetical "comparator nonsignaled program" if they had one additional signal to use. Group differences were assessed using nonparametric Wilcoxon signed rank test.

Results: A total of 202 applicants completed the survey (28% response rate). Most applied to diagnostic radiology (81%). Nearly all respondents used all six gold (98%) and six silver (96.5%) signals. Interview invitation rates were significantly higher for signaled programs (59.8% ± 27.4%) than nonsignaled (8.5% ± 8.5%); the invitation rate at the comparator nonsignaled programs was 37%. Gold-signaled programs had significantly higher interview rates (67.8% ± 29.3) than silver (51.8% ± 31.3%). Respondents used 49.2% (±21.7%) of their signals for "likely to match" programs, 33.1% (±20.9%) for "aspirational" programs, and 17.6% (±15.8%) for "safety" programs. Most respondents (146; 76%) supported continuing the signaling system for future cycles.

Conclusion: Signaling programs significantly enhanced interview invitation rates, with gold signals being more effective than silver. The applicants used about six total signals for "likely-to-match" programs, two for "aspirational" programs, and about four for "safety" programs.

Purpose: To evaluate the extent to which Generative Pre-trained Transformer 4 (GPT-4) can educate patients by generating easily understandable information about the most common interventional radiology (IR) procedures.

Materials and methods: We reviewed 10 IR procedures and prepared prompts for GPT-4 to provide patient educational instructions about each procedure in layman's terms. The instructions were then evaluated by four clinical physicians and nine nonclinical assessors to determine their clinical appropriateness, understandability, and clarity using a survey. A grade-level readability assessment was performed using validated metrics to evaluate accessibility to a wide patient population. The same procedures were also evaluated from the patient instructions available at radiologyinfo.org and compared with GPT-generated instructions utilizing a paired t test.

Results: Evaluation by four clinical physicians shows that nine GPT-generated instructions were fully appropriate, whereas arterial embolization instructions was somewhat appropriate. Evaluation by nine nonclinical assessors shows that paracentesis, dialysis catheter placement, thrombectomy, ultrasound-guided biopsy, and nephrostomy-tube instructions were rated excellent by 57% and good by 43%. The arterial embolization and biliary-drain instructions were rated excellent by 28.6% and good by 71.4%. In contrast, thoracentesis, port placement, and CT-guided biopsy instructions received 43% excellent, 43% good, and 14% fair. The readability assessment across all procedural instructions showed a better Flesch-Kincaid mean grade of GPT-4 instructions compared with radiologyinfo.org (7.8 ± 0.87 versus 9.6 ± 0.83; P = .007) indicating excellent readability at 7th- to 8th-grade level compared with 9th to 10th grade. Additionally there was a lower Gunning Fog mean index (10.4 ± 1.2 versus 12.7 ± 0.93; P = .006), and higher Flesch Reading Ease mean score (69.4 ± 4.8 versus 51.3±3.9; P = .0001) indicating better readability.

Conclusion: IR procedural instructions generated by GPT-4 can aid in improving health literacy and patient-centered care in IR by generating easily understandable explanations.