Coronary Artery Calcification (CAC) is a characteristic pathological alteration in the progression of coronary atherosclerosis and is considered an independent predictor of Major Adverse Cardiovascular Events (MACE). The distribution, pathological classification, and quantitative evaluation of CAC are pivotal factors influencing the incidence of MACE and guiding intracoronary interventions. Deep learning methods, a widely explored domain in artificial intelligence, achieve learning and understanding of big data by constructing multi-layer neural network models. This robust approach offers significant support for intelligent medical image diagnosis within clinical settings. Currently, deep learning methods have been applied to the identification and quantification of coronary artery calcification plaques, which not only improve diagnostic efficiency but also contribute to the early prevention and treatment of patients at moderate to low risk. This article reviews the progress of deep learning applications in coronary artery calcification to gain a comprehensive understanding of this field.
{"title":"Advancements in the application of deep learning for coronary artery calcification","authors":"Ke-Xin Tang, Yan-Lin Wu, Su-Kang Shan, Ling-Qing Yuan","doi":"10.1016/j.metrad.2025.100134","DOIUrl":"10.1016/j.metrad.2025.100134","url":null,"abstract":"<div><div>Coronary Artery Calcification (CAC) is a characteristic pathological alteration in the progression of coronary atherosclerosis and is considered an independent predictor of Major Adverse Cardiovascular Events (MACE). The distribution, pathological classification, and quantitative evaluation of CAC are pivotal factors influencing the incidence of MACE and guiding intracoronary interventions. Deep learning methods, a widely explored domain in artificial intelligence, achieve learning and understanding of big data by constructing multi-layer neural network models. This robust approach offers significant support for intelligent medical image diagnosis within clinical settings. Currently, deep learning methods have been applied to the identification and quantification of coronary artery calcification plaques, which not only improve diagnostic efficiency but also contribute to the early prevention and treatment of patients at moderate to low risk. This article reviews the progress of deep learning applications in coronary artery calcification to gain a comprehensive understanding of this field.</div></div>","PeriodicalId":100921,"journal":{"name":"Meta-Radiology","volume":"3 1","pages":"Article 100134"},"PeriodicalIF":0.0,"publicationDate":"2025-02-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143465256","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-02-02DOI: 10.1016/j.metrad.2025.100135
Jin Liu, Haoting Wang, Lingjiang Li
For mental disorders, the identification of biomarkers with high specificity, sensitivity, and validity remains a major challenge due to their heterogeneity and symptomatic overlap across disorders. In this review, we provide an overview of current research on mental disorders and highlight two key strategies potentially capable of addressing these challenges: data integration and methodological innovation. Effective biomarker identification requires integrating large-scale, multicenter, and multidimensional data integration, including psychological, biological, physiological, and behavioral data. Innovative data acquisition technologies and analytical methods, alongside novel approaches such as leveraging treatment response to validate biomarkers, are equally pivotal for advancing the field. We anticipate that the progress in this domain will be bolstered by the integration of new methodologies and technologies.
{"title":"Rethinking the studies of diagnostic biomarkers for mental disorders","authors":"Jin Liu, Haoting Wang, Lingjiang Li","doi":"10.1016/j.metrad.2025.100135","DOIUrl":"10.1016/j.metrad.2025.100135","url":null,"abstract":"<div><div>For mental disorders, the identification of biomarkers with high specificity, sensitivity, and validity remains a major challenge due to their heterogeneity and symptomatic overlap across disorders. In this review, we provide an overview of current research on mental disorders and highlight two key strategies potentially capable of addressing these challenges: data integration and methodological innovation. Effective biomarker identification requires integrating large-scale, multicenter, and multidimensional data integration, including psychological, biological, physiological, and behavioral data. Innovative data acquisition technologies and analytical methods, alongside novel approaches such as leveraging treatment response to validate biomarkers, are equally pivotal for advancing the field. We anticipate that the progress in this domain will be bolstered by the integration of new methodologies and technologies.</div></div>","PeriodicalId":100921,"journal":{"name":"Meta-Radiology","volume":"3 1","pages":"Article 100135"},"PeriodicalIF":0.0,"publicationDate":"2025-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143403005","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-02DOI: 10.1016/j.metrad.2024.100124
Saher Verma , Leander Maerkisch , Alberto Paderno , Leonard Gilberg , Bianca Teodorescu , Mathias Meyer
In an era where early detection of diseases is paramount, integrating artificial intelligence (AI) into routine lung cancer screening offers a groundbreaking approach to simultaneously uncover multiple health conditions from a single scan. The fact that lung cancer is still the most common cause of cancer-related deaths globally emphasizes how important early detection is to raising survival rates. Traditional low dose computed tomography (LDCT) focuses primarily on identifying lung malignancies, often missing the opportunity to detect other clinically relevant biomarkers. This review explores the expanding role of AI in radiology, where AI-driven algorithms can simultaneously detect multiple biomarkers and composite health measures, facilitating the opportunistic identification of conditions beyond lung cancer. These include musculoskeletal disorders, cardiovascular diseases, pulmonary conditions, hepatic steatosis, and malignancies in the adrenal and thyroid glands, as well as breast tissue. Through an extensive review of current literature sourced from PubMed, the review highlights advancements in AI-driven biomarker detection, evaluates the potential benefits of a broader diagnostic approach, and addresses challenges related to model standardization and clinical integration. AI-enhanced LDCT screening shows significant promise in augmenting routine screenings, potentially advancing early detection, comprehensive patient assessments, and overall disease management across multiple health conditions.
{"title":"One scan, multiple insights: A review of AI-Driven biomarker imaging and composite measure detection in lung cancer screening","authors":"Saher Verma , Leander Maerkisch , Alberto Paderno , Leonard Gilberg , Bianca Teodorescu , Mathias Meyer","doi":"10.1016/j.metrad.2024.100124","DOIUrl":"10.1016/j.metrad.2024.100124","url":null,"abstract":"<div><div>In an era where early detection of diseases is paramount, integrating artificial intelligence (AI) into routine lung cancer screening offers a groundbreaking approach to simultaneously uncover multiple health conditions from a single scan. The fact that lung cancer is still the most common cause of cancer-related deaths globally emphasizes how important early detection is to raising survival rates. Traditional low dose computed tomography (LDCT) focuses primarily on identifying lung malignancies, often missing the opportunity to detect other clinically relevant biomarkers. This review explores the expanding role of AI in radiology, where AI-driven algorithms can simultaneously detect multiple biomarkers and composite health measures, facilitating the opportunistic identification of conditions beyond lung cancer. These include musculoskeletal disorders, cardiovascular diseases, pulmonary conditions, hepatic steatosis, and malignancies in the adrenal and thyroid glands, as well as breast tissue. Through an extensive review of current literature sourced from PubMed, the review highlights advancements in AI-driven biomarker detection, evaluates the potential benefits of a broader diagnostic approach, and addresses challenges related to model standardization and clinical integration. AI-enhanced LDCT screening shows significant promise in augmenting routine screenings, potentially advancing early detection, comprehensive patient assessments, and overall disease management across multiple health conditions.</div></div>","PeriodicalId":100921,"journal":{"name":"Meta-Radiology","volume":"3 1","pages":"Article 100124"},"PeriodicalIF":0.0,"publicationDate":"2025-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143143991","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-01DOI: 10.1016/j.metrad.2024.100112
Ruowei Tang, Pengfei Zhao, Jia Li, Zhixiang Wang, Ning Xu, Zhenchang Wang
The human ear, possessing complex structures like the ossicular chain, cochlea, and auditory nerve, plays a crucial role in hearing and balance. Common ear diseases, such as hearing loss, tinnitus, facial paralysis and vertigo, affect the quality of life of millions in China. Computed tomography (CT) has made significant advancements since its introduction to China in 2000. The resolution improves from millimeter to sub-millimeter levels, and further, to 10 μm through bone-dedicated CT technology. The advancements have made CT become the preferred method for diagnosing various ear conditions, including congenital malformations, trauma, inflammation, and neoplasm. Artificial intelligence (AI) has brought significant breakthroughs in the CT diagnosis. The performance of automatic segmentation of ear structures has dramatically improved with the advent of ultra-high-resolution computed tomography (U-HRCT). AI-driven measurement tools are enhancing the precision and personalization of surgical planning, while deep learning-based anomaly detection is utilized to address the challenges of detecting diverse ear lesions. Furthermore, AI-driven natural language processing and large language models are revolutionizing the generation of radiology reports, providing accurate and standardized diagnostic information. Despite the ongoing challenges, the application of AI in CT is expected to faciliate the otological field, leading to more precise and personalized treatment for ear diseases.
{"title":"Artificial intelligence in CT diagnosis: Current status and future prospects for ear diseases","authors":"Ruowei Tang, Pengfei Zhao, Jia Li, Zhixiang Wang, Ning Xu, Zhenchang Wang","doi":"10.1016/j.metrad.2024.100112","DOIUrl":"10.1016/j.metrad.2024.100112","url":null,"abstract":"<div><div>The human ear, possessing complex structures like the ossicular chain, cochlea, and auditory nerve, plays a crucial role in hearing and balance. Common ear diseases, such as hearing loss, tinnitus, facial paralysis and vertigo, affect the quality of life of millions in China. Computed tomography (CT) has made significant advancements since its introduction to China in 2000. The resolution improves from millimeter to sub-millimeter levels, and further, to 10 μm through bone-dedicated CT technology. The advancements have made CT become the preferred method for diagnosing various ear conditions, including congenital malformations, trauma, inflammation, and neoplasm. Artificial intelligence (AI) has brought significant breakthroughs in the CT diagnosis. The performance of automatic segmentation of ear structures has dramatically improved with the advent of ultra-high-resolution computed tomography (U-HRCT). AI-driven measurement tools are enhancing the precision and personalization of surgical planning, while deep learning-based anomaly detection is utilized to address the challenges of detecting diverse ear lesions. Furthermore, AI-driven natural language processing and large language models are revolutionizing the generation of radiology reports, providing accurate and standardized diagnostic information. Despite the ongoing challenges, the application of AI in CT is expected to faciliate the otological field, leading to more precise and personalized treatment for ear diseases.</div></div>","PeriodicalId":100921,"journal":{"name":"Meta-Radiology","volume":"2 4","pages":"Article 100112"},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142748191","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-01DOI: 10.1016/j.metrad.2024.100099
Yunyi Liu , Yingshu Li , Zhanyu Wang , Xinyu Liang , Lingqiao Liu , Lei Wang , Leyang Cui , Zhaopeng Tu , Longyue Wang , Luping Zhou
This work evaluates GPT-4V's multimodal capability for medical image analysis, focusing on three representative tasks radiology report generation, medical visual question answering, and medical visual grounding. For the evaluation, a set of prompts is designed for each task to induce the corresponding capability of GPT-4V to produce sufficiently good outputs. Three evaluation ways including quantitative analysis, human evaluation, and case study are employed to achieve an in-depth and extensive evaluation. Our evaluation shows that GPT-4V excels in understanding medical images can generate high-quality radiology reports and effectively answer questions about medical images. Meanwhile, it is found that its performance for medical visual grounding needs to be substantially improved. In addition, we observe the discrepancy between the evaluation outcome from quantitative analysis and that from human evaluation. This discrepancy suggests the limitations of conventional metrics in assessing the performance of large language models like GPT-4V and the necessity of developing new metrics for automatic quantitative analysis.
{"title":"A systematic evaluation of GPT-4V's multimodal capability for chest X-ray image analysis","authors":"Yunyi Liu , Yingshu Li , Zhanyu Wang , Xinyu Liang , Lingqiao Liu , Lei Wang , Leyang Cui , Zhaopeng Tu , Longyue Wang , Luping Zhou","doi":"10.1016/j.metrad.2024.100099","DOIUrl":"10.1016/j.metrad.2024.100099","url":null,"abstract":"<div><div>This work evaluates GPT-4V's multimodal capability for medical image analysis, focusing on three representative tasks radiology report generation, medical visual question answering, and medical visual grounding. For the evaluation, a set of prompts is designed for each task to induce the corresponding capability of GPT-4V to produce sufficiently good outputs. Three evaluation ways including quantitative analysis, human evaluation, and case study are employed to achieve an in-depth and extensive evaluation. Our evaluation shows that GPT-4V excels in understanding medical images can generate high-quality radiology reports and effectively answer questions about medical images. Meanwhile, it is found that its performance for medical visual grounding needs to be substantially improved. In addition, we observe the discrepancy between the evaluation outcome from quantitative analysis and that from human evaluation. This discrepancy suggests the limitations of conventional metrics in assessing the performance of large language models like GPT-4V and the necessity of developing new metrics for automatic quantitative analysis.</div></div>","PeriodicalId":100921,"journal":{"name":"Meta-Radiology","volume":"2 4","pages":"Article 100099"},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141711609","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-12-01DOI: 10.1016/j.metrad.2024.100113
Xinrui Song , Jiajin Zhang , Pingkun Yan, Juergen Hahn, Uwe Kruger, Hisham Mohamed, Ge Wang
The integration of artificial intelligence (AI) chatbots into higher education marks a shift towards a new generation of pedagogical tools, mirroring the arrival of milestones like the internet. With the launch of ChatGPT-4 Turbo in November 2023, we developed a ChatGPT-based teaching application (https://chat.openai.com/g/g-1imx1py4K-chatge-medical-imaging) and integrated it into our undergraduate medical imaging course in the Spring 2024 semester. This study investigates the use of ChatGPT throughout a semester-long trial, providing insights into students' engagement, perception, and the overall educational effectiveness of the technology. We systematically collected and analyzed data concerning students’ interaction with ChatGPT, focusing on their attitudes, concerns, and usage patterns. The findings indicate that ChatGPT offers significant advantages such as improved information access and increased interactivity, but its adoption is accompanied by concerns about the accuracy of the information provided and the necessity for well-defined guidelines to optimize its use.
{"title":"Integrating AI in college education: Positive yet mixed experiences with ChatGPT","authors":"Xinrui Song , Jiajin Zhang , Pingkun Yan, Juergen Hahn, Uwe Kruger, Hisham Mohamed, Ge Wang","doi":"10.1016/j.metrad.2024.100113","DOIUrl":"10.1016/j.metrad.2024.100113","url":null,"abstract":"<div><div>The integration of artificial intelligence (AI) chatbots into higher education marks a shift towards a new generation of pedagogical tools, mirroring the arrival of milestones like the internet. With the launch of ChatGPT-4 Turbo in November 2023, we developed a ChatGPT-based teaching application (<span><span>https://chat.openai.com/g/g-1imx1py4K-chatge-medical-imaging</span><svg><path></path></svg></span>) and integrated it into our undergraduate medical imaging course in the Spring 2024 semester. This study investigates the use of ChatGPT throughout a semester-long trial, providing insights into students' engagement, perception, and the overall educational effectiveness of the technology. We systematically collected and analyzed data concerning students’ interaction with ChatGPT, focusing on their attitudes, concerns, and usage patterns. The findings indicate that ChatGPT offers significant advantages such as improved information access and increased interactivity, but its adoption is accompanied by concerns about the accuracy of the information provided and the necessity for well-defined guidelines to optimize its use.</div></div>","PeriodicalId":100921,"journal":{"name":"Meta-Radiology","volume":"2 4","pages":"Article 100113"},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142748190","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-22DOI: 10.1016/j.metrad.2024.100114
Li-Miao Zou, Ke-Ting Xu, Yi-Ning Wang
Coronary artery disease (CAD) remains the leading cause of morbidity and mortality globally. The recent years have witnessed a steep increase in the number of cardiac CT examinations, including coronary CT angiography (CCTA) and non-contrast ECG-gated cardiac CT, which put a heavy load on the radiologists. Artificial intelligence (AI), which aims to automate tasks that resembles human intelligence, presents itself as a promising solution. AI has played an increasingly important role in the field of cardiac CT, from advanced image reconstruction to coronary stenosis and plaque analysis, predicting flow, and potentially better risk stratification and event prediction. In this review, we aim to summarize state-of-the-art AI approaches applied to cardiac CT and their future implications.
{"title":"Research advances and applications of artificial intelligence in cardiac CT","authors":"Li-Miao Zou, Ke-Ting Xu, Yi-Ning Wang","doi":"10.1016/j.metrad.2024.100114","DOIUrl":"10.1016/j.metrad.2024.100114","url":null,"abstract":"<div><div>Coronary artery disease (CAD) remains the leading cause of morbidity and mortality globally. The recent years have witnessed a steep increase in the number of cardiac CT examinations, including coronary CT angiography (CCTA) and non-contrast ECG-gated cardiac CT, which put a heavy load on the radiologists. Artificial intelligence (AI), which aims to automate tasks that resembles human intelligence, presents itself as a promising solution. AI has played an increasingly important role in the field of cardiac CT, from advanced image reconstruction to coronary stenosis and plaque analysis, predicting flow, and potentially better risk stratification and event prediction. In this review, we aim to summarize state-of-the-art AI approaches applied to cardiac CT and their future implications.</div></div>","PeriodicalId":100921,"journal":{"name":"Meta-Radiology","volume":"2 4","pages":"Article 100114"},"PeriodicalIF":0.0,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142593666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-01DOI: 10.1016/j.metrad.2024.100111
Boyang Mao , Hong Wang , Hongxi Zhang , Xueliang Shang , Zhi Yang
Background
The corpus callosum plays a crucial role in integrated brain functions, and its development in childhood is strongly associated with subsequent cognitive, emotional, and behavioral development. However, there is still a lack of clear understanding regarding the developmental trends of the corpus callosum in preschool children. This study aims to comprehensively investigate age and sex differences in the thickness of the corpus callosum in typical developing children between 1 and 6 years old.
Methods
T1-weighted structural MRI data were collected from a sample of 295 neurologically normal children aged 1–6 years. Utilizing the specialized corpus callosum segmentation software Yuki, thickness measurements of the mid-sagittal plane of the corpus callosum were obtained.
Results
The anterior part exhibited faster growth compared to the middle and posterior sections, while growth at the extremities was not statistically significant. Furthermore, gender differences were identified, with males showing earlier development of the corpus callosum, particularly between ages 1 and 3. Conversely, females exhibited the most notable increase in thickness between ages 3 and 5.
Conclusion
This study provides significant insights into the developmental trends of the mid-sagittal plane of the corpus callosum in preschool children. It reveals distinct non-linear developmental patterns in different sections of the corpus callosum and highlights the influence of sex on these developmental patterns.
{"title":"Developmental trends in corpus callosum thickness among preschool children","authors":"Boyang Mao , Hong Wang , Hongxi Zhang , Xueliang Shang , Zhi Yang","doi":"10.1016/j.metrad.2024.100111","DOIUrl":"10.1016/j.metrad.2024.100111","url":null,"abstract":"<div><h3>Background</h3><div>The corpus callosum plays a crucial role in integrated brain functions, and its development in childhood is strongly associated with subsequent cognitive, emotional, and behavioral development. However, there is still a lack of clear understanding regarding the developmental trends of the corpus callosum in preschool children. This study aims to comprehensively investigate age and sex differences in the thickness of the corpus callosum in typical developing children between 1 and 6 years old.</div></div><div><h3>Methods</h3><div>T1-weighted structural MRI data were collected from a sample of 295 neurologically normal children aged 1–6 years. Utilizing the specialized corpus callosum segmentation software Yuki, thickness measurements of the mid-sagittal plane of the corpus callosum were obtained.</div></div><div><h3>Results</h3><div>The anterior part exhibited faster growth compared to the middle and posterior sections, while growth at the extremities was not statistically significant. Furthermore, gender differences were identified, with males showing earlier development of the corpus callosum, particularly between ages 1 and 3. Conversely, females exhibited the most notable increase in thickness between ages 3 and 5.</div></div><div><h3>Conclusion</h3><div>This study provides significant insights into the developmental trends of the mid-sagittal plane of the corpus callosum in preschool children. It reveals distinct non-linear developmental patterns in different sections of the corpus callosum and highlights the influence of sex on these developmental patterns.</div></div>","PeriodicalId":100921,"journal":{"name":"Meta-Radiology","volume":"2 4","pages":"Article 100111"},"PeriodicalIF":0.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142651900","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-21DOI: 10.1016/j.metrad.2024.100103
Yutong Zhang , Yi Pan , Tianyang Zhong , Peixin Dong , Kangni Xie , Yuxiao Liu , Hanqi Jiang , Zihao Wu , Zhengliang Liu , Wei Zhao , Wei Zhang , Shijie Zhao , Tuo Zhang , Xi Jiang , Dinggang Shen , Tianming Liu , Xin Zhang
Medical images and radiology reports are essential for physicians to diagnose medical conditions. However, the vast diversity and cross-source heterogeneity inherent in these data have posed significant challenges to the generalizability of current data-mining methods for clinical decision-making. Recently, multimodal large language models (MLLMs), especially Gemini-Vision-series (Gemini) and GPT-4-series (GPT-4) models, have revolutionized numerous domains, significantly impacting the medical field. In this study, we conducted a detailed evaluation of the performance of the Gemini series models (including Gemini-1.0-Pro-Vision, Gemini-1.5-Pro, and Gemini-1.5-Flash) and GPT series models (including GPT-4o, GPT-4-Turbo, and GPT-3.5-Turbo) across 14 medical datasets, covering 5 medical imaging categories (dermatology, radiology, dentistry, ophthalmology, and endoscopy) and 3 radiology report datasets. The investigated tasks encompass disease classification, lesion segmentation, anatomical localization, disease diagnosis, report generation, and lesion detection. Moreover, we also validated the performance of the Claude-3-Opus, Yi-Large, Yi-Large-Turbo, and LLaMA 3 models to gain a comprehensive understanding of the MLLM models in the medical field. Our experimental results demonstrated that Gemini-series models excelled in report generation and lesion detection but faces challenges in disease classification and anatomical localization. Conversely, GPT-series models exhibited proficiency in lesion segmentation and anatomical localization but encountered difficulties in disease diagnosis and lesion detection. Additionally, both the Gemini series and GPT series contain models that have demonstrated commendable generation efficiency. While both models hold promise in reducing physician workload, alleviating pressure on limited healthcare resources, and fostering collaboration between clinical practitioners and artificial intelligence technologies, substantial enhancements and comprehensive validations remain imperative before clinical deployment.
{"title":"Potential of multimodal large language models for data mining of medical images and free-text reports","authors":"Yutong Zhang , Yi Pan , Tianyang Zhong , Peixin Dong , Kangni Xie , Yuxiao Liu , Hanqi Jiang , Zihao Wu , Zhengliang Liu , Wei Zhao , Wei Zhang , Shijie Zhao , Tuo Zhang , Xi Jiang , Dinggang Shen , Tianming Liu , Xin Zhang","doi":"10.1016/j.metrad.2024.100103","DOIUrl":"10.1016/j.metrad.2024.100103","url":null,"abstract":"<div><div>Medical images and radiology reports are essential for physicians to diagnose medical conditions. However, the vast diversity and cross-source heterogeneity inherent in these data have posed significant challenges to the generalizability of current data-mining methods for clinical decision-making. Recently, multimodal large language models (MLLMs), especially Gemini-Vision-series (Gemini) and GPT-4-series (GPT-4) models, have revolutionized numerous domains, significantly impacting the medical field. In this study, we conducted a detailed evaluation of the performance of the Gemini series models (including Gemini-1.0-Pro-Vision, Gemini-1.5-Pro, and Gemini-1.5-Flash) and GPT series models (including GPT-4o, GPT-4-Turbo, and GPT-3.5-Turbo) across 14 medical datasets, covering 5 medical imaging categories (dermatology, radiology, dentistry, ophthalmology, and endoscopy) and 3 radiology report datasets. The investigated tasks encompass disease classification, lesion segmentation, anatomical localization, disease diagnosis, report generation, and lesion detection. Moreover, we also validated the performance of the Claude-3-Opus, Yi-Large, Yi-Large-Turbo, and LLaMA 3 models to gain a comprehensive understanding of the MLLM models in the medical field. Our experimental results demonstrated that Gemini-series models excelled in report generation and lesion detection but faces challenges in disease classification and anatomical localization. Conversely, GPT-series models exhibited proficiency in lesion segmentation and anatomical localization but encountered difficulties in disease diagnosis and lesion detection. Additionally, both the Gemini series and GPT series contain models that have demonstrated commendable generation efficiency. While both models hold promise in reducing physician workload, alleviating pressure on limited healthcare resources, and fostering collaboration between clinical practitioners and artificial intelligence technologies, substantial enhancements and comprehensive validations remain imperative before clinical deployment.</div></div>","PeriodicalId":100921,"journal":{"name":"Meta-Radiology","volume":"2 4","pages":"Article 100103"},"PeriodicalIF":0.0,"publicationDate":"2024-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142651946","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-18DOI: 10.1016/j.metrad.2024.100102
Zhifeng Wang, Renjiao Yi, Xin Wen, Chenyang Zhu, Kai Xu
With the rapid development of 3D vision and computer graphics technology, the way humans interact with the world has undergone significant transformations. 3D vision-related technologies have profoundly impacted the analysis of cardiovascular diseases (CVD) based on medical imaging diagnosis. In this paper, we provide a comprehensive review of CVD analysis based on 3D vision. First, we delineate cardiovascular imaging and cardiovascular data types from both medical and computational perspectives. Then, we introduce a systematic taxonomy to comprehensively review the current practices of 3D vision in cardiovascular applications, covering aspects such as 3D vascular segmentation, 3D vascular map generation, 3D vascular reconstruction, and 3D vascular super-resolution. Additionally, we compile a list of publicly accessible cardiac image datasets and code repositories to support the reproduction of related algorithms and foster data and algorithm sharing within the community. Finally, we discuss the inherent challenges and limitations of cardiovascular imaging methods based on 3D vision and their potential and propose directions for overcoming these obstacles in future research.
{"title":"Cardiovascular medical image and analysis based on 3D vision: A comprehensive survey","authors":"Zhifeng Wang, Renjiao Yi, Xin Wen, Chenyang Zhu, Kai Xu","doi":"10.1016/j.metrad.2024.100102","DOIUrl":"10.1016/j.metrad.2024.100102","url":null,"abstract":"<div><div>With the rapid development of 3D vision and computer graphics technology, the way humans interact with the world has undergone significant transformations. 3D vision-related technologies have profoundly impacted the analysis of cardiovascular diseases (CVD) based on medical imaging diagnosis. In this paper, we provide a comprehensive review of CVD analysis based on 3D vision. First, we delineate cardiovascular imaging and cardiovascular data types from both medical and computational perspectives. Then, we introduce a systematic taxonomy to comprehensively review the current practices of 3D vision in cardiovascular applications, covering aspects such as 3D vascular segmentation, 3D vascular map generation, 3D vascular reconstruction, and 3D vascular super-resolution. Additionally, we compile a list of publicly accessible cardiac image datasets and code repositories to support the reproduction of related algorithms and foster data and algorithm sharing within the community. Finally, we discuss the inherent challenges and limitations of cardiovascular imaging methods based on 3D vision and their potential and propose directions for overcoming these obstacles in future research.</div></div>","PeriodicalId":100921,"journal":{"name":"Meta-Radiology","volume":"2 4","pages":"Article 100102"},"PeriodicalIF":0.0,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142651968","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号