Orhun Utku Aydin , Adam Hilbert , Alexander Koch , Felix Lohrke , Jana Rieger , Satoru Tanioka , Dietmar Frey
{"title":"使用 3D-StyleGAN 建立威利斯环的生成模型。","authors":"Orhun Utku Aydin , Adam Hilbert , Alexander Koch , Felix Lohrke , Jana Rieger , Satoru Tanioka , Dietmar Frey","doi":"10.1016/j.neuroimage.2024.120936","DOIUrl":null,"url":null,"abstract":"<div><div>The circle of Willis (CoW) is a network of cerebral arteries with significant inter-individual anatomical variations. Deep learning has been used to characterize and quantify the status of the CoW in various applications for the diagnosis and treatment of cerebrovascular disease. In medical imaging, the performance of deep learning models is limited by the diversity and size of training datasets. To address medical data scarcity, generative AI models have been applied to generate synthetic vessel neuroimaging data. However, the proposed methods produce synthetic data with limited anatomical fidelity or downstream utility in tasks concerning vessel characteristics.</div><div>We adapted the StyleGANv2 architecture to 3D to synthesize Time-of-Flight Magnetic Resonance Angiography (TOF MRA) volumes of the CoW. For generative modeling, we used 1782 individual TOF MRA scans from 6 open source datasets. To train the adapted 3D StyleGAN model with limited data we employed differentiable data augmentations, used mixed precision and a cropped region of interest of size 32 × 128 × 128 to tackle computational constraints. The performance was evaluated quantitatively using the Fréchet Inception Distance (FID), MedicalNet distance (MD) and Area Under the Curve of the Precision and Recall Curve for Distributions (AUC-PRD). Qualitative analysis was performed via a visual Turing test. We demonstrated the utility of generated data in a downstream task of multiclass semantic segmentation of CoW arteries. Vessel segmentation performance was assessed quantitatively using the Dice coefficient and the Hausdorff distance.</div><div>The best-performing 3D StyleGANv2 architecture generated high-quality and diverse synthetic TOF MRA volumes (FID: 12.17, MD: 0.00078, AUC-PRD: 0.9610). Multiclass vessel segmentation models trained on synthetic data alone achieved comparable performance to models trained using real data in most arteries. The addition of synthetic data to a baseline training set improved segmentation performance in underrepresented artery segments, similar to the addition of real data.</div><div>In conclusion, generative modeling of the Circle of Willis via synthesis of 3D TOF MRA data paves the way for generalizable deep learning applications in cerebrovascular disease. In the future, the extensions of the provided methodology to other medical imaging problems or modalities with the inclusion of pathological datasets has the potential to advance the development of more robust AI models for clinical applications.</div></div>","PeriodicalId":19299,"journal":{"name":"NeuroImage","volume":"304 ","pages":"Article 120936"},"PeriodicalIF":4.7000,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Generative modeling of the Circle of Willis using 3D-StyleGAN\",\"authors\":\"Orhun Utku Aydin , Adam Hilbert , Alexander Koch , Felix Lohrke , Jana Rieger , Satoru Tanioka , Dietmar Frey\",\"doi\":\"10.1016/j.neuroimage.2024.120936\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>The circle of Willis (CoW) is a network of cerebral arteries with significant inter-individual anatomical variations. Deep learning has been used to characterize and quantify the status of the CoW in various applications for the diagnosis and treatment of cerebrovascular disease. In medical imaging, the performance of deep learning models is limited by the diversity and size of training datasets. To address medical data scarcity, generative AI models have been applied to generate synthetic vessel neuroimaging data. However, the proposed methods produce synthetic data with limited anatomical fidelity or downstream utility in tasks concerning vessel characteristics.</div><div>We adapted the StyleGANv2 architecture to 3D to synthesize Time-of-Flight Magnetic Resonance Angiography (TOF MRA) volumes of the CoW. For generative modeling, we used 1782 individual TOF MRA scans from 6 open source datasets. To train the adapted 3D StyleGAN model with limited data we employed differentiable data augmentations, used mixed precision and a cropped region of interest of size 32 × 128 × 128 to tackle computational constraints. The performance was evaluated quantitatively using the Fréchet Inception Distance (FID), MedicalNet distance (MD) and Area Under the Curve of the Precision and Recall Curve for Distributions (AUC-PRD). Qualitative analysis was performed via a visual Turing test. We demonstrated the utility of generated data in a downstream task of multiclass semantic segmentation of CoW arteries. Vessel segmentation performance was assessed quantitatively using the Dice coefficient and the Hausdorff distance.</div><div>The best-performing 3D StyleGANv2 architecture generated high-quality and diverse synthetic TOF MRA volumes (FID: 12.17, MD: 0.00078, AUC-PRD: 0.9610). Multiclass vessel segmentation models trained on synthetic data alone achieved comparable performance to models trained using real data in most arteries. The addition of synthetic data to a baseline training set improved segmentation performance in underrepresented artery segments, similar to the addition of real data.</div><div>In conclusion, generative modeling of the Circle of Willis via synthesis of 3D TOF MRA data paves the way for generalizable deep learning applications in cerebrovascular disease. In the future, the extensions of the provided methodology to other medical imaging problems or modalities with the inclusion of pathological datasets has the potential to advance the development of more robust AI models for clinical applications.</div></div>\",\"PeriodicalId\":19299,\"journal\":{\"name\":\"NeuroImage\",\"volume\":\"304 \",\"pages\":\"Article 120936\"},\"PeriodicalIF\":4.7000,\"publicationDate\":\"2024-11-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"NeuroImage\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1053811924004336\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"NEUROIMAGING\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"NeuroImage","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1053811924004336","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"NEUROIMAGING","Score":null,"Total":0}
Generative modeling of the Circle of Willis using 3D-StyleGAN
The circle of Willis (CoW) is a network of cerebral arteries with significant inter-individual anatomical variations. Deep learning has been used to characterize and quantify the status of the CoW in various applications for the diagnosis and treatment of cerebrovascular disease. In medical imaging, the performance of deep learning models is limited by the diversity and size of training datasets. To address medical data scarcity, generative AI models have been applied to generate synthetic vessel neuroimaging data. However, the proposed methods produce synthetic data with limited anatomical fidelity or downstream utility in tasks concerning vessel characteristics.
We adapted the StyleGANv2 architecture to 3D to synthesize Time-of-Flight Magnetic Resonance Angiography (TOF MRA) volumes of the CoW. For generative modeling, we used 1782 individual TOF MRA scans from 6 open source datasets. To train the adapted 3D StyleGAN model with limited data we employed differentiable data augmentations, used mixed precision and a cropped region of interest of size 32 × 128 × 128 to tackle computational constraints. The performance was evaluated quantitatively using the Fréchet Inception Distance (FID), MedicalNet distance (MD) and Area Under the Curve of the Precision and Recall Curve for Distributions (AUC-PRD). Qualitative analysis was performed via a visual Turing test. We demonstrated the utility of generated data in a downstream task of multiclass semantic segmentation of CoW arteries. Vessel segmentation performance was assessed quantitatively using the Dice coefficient and the Hausdorff distance.
The best-performing 3D StyleGANv2 architecture generated high-quality and diverse synthetic TOF MRA volumes (FID: 12.17, MD: 0.00078, AUC-PRD: 0.9610). Multiclass vessel segmentation models trained on synthetic data alone achieved comparable performance to models trained using real data in most arteries. The addition of synthetic data to a baseline training set improved segmentation performance in underrepresented artery segments, similar to the addition of real data.
In conclusion, generative modeling of the Circle of Willis via synthesis of 3D TOF MRA data paves the way for generalizable deep learning applications in cerebrovascular disease. In the future, the extensions of the provided methodology to other medical imaging problems or modalities with the inclusion of pathological datasets has the potential to advance the development of more robust AI models for clinical applications.
期刊介绍:
NeuroImage, a Journal of Brain Function provides a vehicle for communicating important advances in acquiring, analyzing, and modelling neuroimaging data and in applying these techniques to the study of structure-function and brain-behavior relationships. Though the emphasis is on the macroscopic level of human brain organization, meso-and microscopic neuroimaging across all species will be considered if informative for understanding the aforementioned relationships.