Radiology最新文献

Background Rapid advances in large language models (LLMs) have led to the development of numerous commercial and open-source models. While recent publications have explored OpenAI's GPT-4 to extract information of interest from radiology reports, there has not been a real-world comparison of GPT-4 to leading open-source models. Purpose To compare different leading open-source LLMs to GPT-4 on the task of extracting relevant findings from chest radiograph reports. Materials and Methods Two independent datasets of free-text radiology reports from chest radiograph examinations were used in this retrospective study performed between February 2, 2024, and February 14, 2024. The first dataset consisted of reports from the ImaGenome dataset, providing reference standard annotations from the MIMIC-CXR database acquired between 2011 and 2016. The second dataset consisted of randomly selected reports created at the Massachusetts General Hospital between July 2019 and July 2021. In both datasets, the commercial models GPT-3.5 Turbo and GPT-4 were compared with open-source models that included Mistral-7B and Mixtral-8 × 7B (Mistral AI), Llama 2-13B and Llama 2-70B (Meta), and Qwen1.5-72B (Alibaba Group), as well as CheXbert and CheXpert-labeler (Stanford ML Group), in their ability to accurately label the presence of multiple findings in radiograph text reports using zero-shot and few-shot prompting. The McNemar test was used to compare F1 scores between models. Results On the ImaGenome dataset (n = 450), the open-source model with the highest score, Llama 2-70B, achieved micro F1 scores of 0.97 and 0.97 for zero-shot and few-shot prompting, respectively, compared with the GPT-4 F1 scores of 0.98 and 0.98 (P > .99 and < .001 for superiority of GPT-4). On the institutional dataset (n = 500), the open-source model with the highest score, an ensemble model, achieved micro F1 scores of 0.96 and 0.97 for zero-shot and few-shot prompting, respectively, compared with the GPT-4 F1 scores of 0.98 and 0.97 (P < .001 and > .99 for superiority of GPT-4). Conclusion Although GPT-4 was superior to open-source models in zero-shot report labeling, few-shot prompting with a small number of example reports closely matched the performance of GPT-4. The benefit of few-shot prompting varied across datasets and models. © RSNA, 2024 Supplemental material is available for this article.

Background: MR spectroscopic imaging (MRSI) can be used to quantify an extended brain metabolic profile but is confounded by changes in tissue water levels due to disease.

Purpose: To develop a fast absolute quantification method for metabolite concentrations combining whole-brain MRSI with echo-planar time-resolved imaging (EPTI) relaxometry in individuals with glioma and healthy individuals.

Materials and methods: In this prospective study performed from August 2022 to August 2023, using internal water as concentration reference, the MRSI-EPTI quantification method was compared with the conventional method using population-average literature relaxation values. Healthy participants and participants with mutant IDH1 gliomas underwent imaging at 3 T with a 32-channel coil. Real-time navigated adiabatic spiral three-dimensional MRSI scans were acquired in approximately 8 minutes and reconstructed with a super-resolution pipeline to obtain brain metabolic images at 2.4-mm isotropic resolution. High-spatial-resolution multiparametric EPTI was performed in 3 minutes, with 1-mm isotropic resolution, to correct the relaxation and proton density of the water reference signal. Bland-Altman analysis and the Wilcoxon signed rank test were used to compare absolute quantifications from the proposed and conventional methods.

Results: Six healthy participants (four male; mean age, 37 years ± 11 [SD]) and nine participants with glioma (six male; mean age, 41 years ± 15; one with wild-type IDH1 and eight with mutant IDH1) were included. In healthy participants, there was good agreement (+4% bias) between metabolic concentrations derived using the two methods, with a CI of plus or minus 26%. In participants with glioma, there was large disagreement between the two methods (+39% bias) and a CI of plus or minus 55%. The proposed quantification method improved tumor contrast-to-noise ratio (median values) for total N-acetyl-aspartate (EPTI: 0.541 [95% CI: 0.217, 0.910]; conventional: 0.484 [95% CI: 0.199, 0.823]), total choline (EPTI: 1.053 [95% CI: 0.681, 1.713]; conventional: 0.940 [95% CI: 0.617, 1.295]), and total creatine (EPTI: 0.745 [95% CI: 0.628, 0.909]; conventional: 0.553 [95% CI: 0.444, 0.828]) (P = .03 for all).

Conclusion: The whole-brain MRSI-EPTI method provided fast absolute quantification of metabolic concentrations with individual-specific corrections at 2.4-mm isotropic resolution, yielding concentrations closer to the true value in disease than the conventional literature-based corrections. © RSNA, 2024 Supplemental material is available for this article.