Pub Date : 2022-08-24DOI: 10.48550/arXiv.2208.11669
Dimitris Stripelis, Umang Gupta, N. Dhinagar, G. V. Steeg, Paul M. Thompson, J. Ambite
Federated training of large deep neural networks can often be restrictive due to the increasing costs of communicating the updates with increasing model sizes. Various model pruning techniques have been designed in centralized settings to reduce inference times. Combining centralized pruning techniques with federated training seems intuitive for reducing communication costs -- by pruning the model parameters right before the communication step. Moreover, such a progressive model pruning approach during training can also reduce training times/costs. To this end, we propose FedSparsify, which performs model pruning during federated training. In our experiments in centralized and federated settings on the brain age prediction task (estimating a person's age from their brain MRI), we demonstrate that models can be pruned up to 95% sparsity without affecting performance even in challenging federated learning environments with highly heterogeneous data distributions. One surprising benefit of model pruning is improved model privacy. We demonstrate that models with high sparsity are less susceptible to membership inference attacks, a type of privacy attack.
{"title":"Towards Sparsified Federated Neuroimaging Models via Weight Pruning","authors":"Dimitris Stripelis, Umang Gupta, N. Dhinagar, G. V. Steeg, Paul M. Thompson, J. Ambite","doi":"10.48550/arXiv.2208.11669","DOIUrl":"https://doi.org/10.48550/arXiv.2208.11669","url":null,"abstract":"Federated training of large deep neural networks can often be restrictive due to the increasing costs of communicating the updates with increasing model sizes. Various model pruning techniques have been designed in centralized settings to reduce inference times. Combining centralized pruning techniques with federated training seems intuitive for reducing communication costs -- by pruning the model parameters right before the communication step. Moreover, such a progressive model pruning approach during training can also reduce training times/costs. To this end, we propose FedSparsify, which performs model pruning during federated training. In our experiments in centralized and federated settings on the brain age prediction task (estimating a person's age from their brain MRI), we demonstrate that models can be pruned up to 95% sparsity without affecting performance even in challenging federated learning environments with highly heterogeneous data distributions. One surprising benefit of model pruning is improved model privacy. We demonstrate that models with high sparsity are less susceptible to membership inference attacks, a type of privacy attack.","PeriodicalId":347091,"journal":{"name":"DeCaF/FAIR@MICCAI","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-08-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114183578","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-08-22DOI: 10.48550/arXiv.2208.10553
H. Roth, Ali Hatamizadeh, Ziyue Xu, Can Zhao, Wenqi Li, A. Myronenko, Daguang Xu
Split learning (SL) has been proposed to train deep learning models in a decentralized manner. For decentralized healthcare applications with vertical data partitioning, SL can be beneficial as it allows institutes with complementary features or images for a shared set of patients to jointly develop more robust and generalizable models. In this work, we propose"Split-U-Net"and successfully apply SL for collaborative biomedical image segmentation. Nonetheless, SL requires the exchanging of intermediate activation maps and gradients to allow training models across different feature spaces, which might leak data and raise privacy concerns. Therefore, we also quantify the amount of data leakage in common SL scenarios for biomedical image segmentation and provide ways to counteract such leakage by applying appropriate defense strategies.
{"title":"Split-U-Net: Preventing Data Leakage in Split Learning for Collaborative Multi-Modal Brain Tumor Segmentation","authors":"H. Roth, Ali Hatamizadeh, Ziyue Xu, Can Zhao, Wenqi Li, A. Myronenko, Daguang Xu","doi":"10.48550/arXiv.2208.10553","DOIUrl":"https://doi.org/10.48550/arXiv.2208.10553","url":null,"abstract":"Split learning (SL) has been proposed to train deep learning models in a decentralized manner. For decentralized healthcare applications with vertical data partitioning, SL can be beneficial as it allows institutes with complementary features or images for a shared set of patients to jointly develop more robust and generalizable models. In this work, we propose\"Split-U-Net\"and successfully apply SL for collaborative biomedical image segmentation. Nonetheless, SL requires the exchanging of intermediate activation maps and gradients to allow training models across different feature spaces, which might leak data and raise privacy concerns. Therefore, we also quantify the amount of data leakage in common SL scenarios for biomedical image segmentation and provide ways to counteract such leakage by applying appropriate defense strategies.","PeriodicalId":347091,"journal":{"name":"DeCaF/FAIR@MICCAI","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134382767","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-08-21DOI: 10.48550/arXiv.2208.10919
S. M. Hosseini, Milad Sikaroudi, Morteza Babaie, H. Tizhoosh
. Federated learning (FL) is a decentralized method enabling hospitals to collaboratively learn a model without sharing private pa-tient data for training. In FL, participant hospitals periodically exchange training results rather than training samples with a central server. How-ever, having access to model parameters or gradients can expose private training data samples. To address this challenge, we adopt secure multiparty computation (SMC) to establish a privacy-preserving federated learning framework. In our proposed method, the hospitals are divided into clusters. After local training, each hospital splits its model weights among other hospitals in the same cluster such that no single hospital can retrieve other hospitals’ weights on its own. Then, all hospitals sum up the received weights, sending the results to the central server. Fi-nally, the central server aggregates the results, retrieving the average of models’ weights and updating the model without having access to individual hospitals’ weights. We conduct experiments on a publicly available repository, The Cancer Genome Atlas (TCGA). We compare the performance of the proposed framework with differential privacy and federated averaging as the baseline. The results reveal that compared to differential privacy, our framework can achieve higher accuracy with no privacy leakage risk at a cost of higher communication overhead.
{"title":"Cluster Based Secure Multi-Party Computation in Federated Learning for Histopathology Images","authors":"S. M. Hosseini, Milad Sikaroudi, Morteza Babaie, H. Tizhoosh","doi":"10.48550/arXiv.2208.10919","DOIUrl":"https://doi.org/10.48550/arXiv.2208.10919","url":null,"abstract":". Federated learning (FL) is a decentralized method enabling hospitals to collaboratively learn a model without sharing private pa-tient data for training. In FL, participant hospitals periodically exchange training results rather than training samples with a central server. How-ever, having access to model parameters or gradients can expose private training data samples. To address this challenge, we adopt secure multiparty computation (SMC) to establish a privacy-preserving federated learning framework. In our proposed method, the hospitals are divided into clusters. After local training, each hospital splits its model weights among other hospitals in the same cluster such that no single hospital can retrieve other hospitals’ weights on its own. Then, all hospitals sum up the received weights, sending the results to the central server. Fi-nally, the central server aggregates the results, retrieving the average of models’ weights and updating the model without having access to individual hospitals’ weights. We conduct experiments on a publicly available repository, The Cancer Genome Atlas (TCGA). We compare the performance of the proposed framework with differential privacy and federated averaging as the baseline. The results reveal that compared to differential privacy, our framework can achieve higher accuracy with no privacy leakage risk at a cost of higher communication overhead.","PeriodicalId":347091,"journal":{"name":"DeCaF/FAIR@MICCAI","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129642494","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-08-05DOI: 10.48550/arXiv.2208.03305
Germain Morilhat, Naomi Kifle, Sandy FinesilverSmith, B. Ruijsink, V. Vergani, Habtamu Tegegne Desita, Z. Desita, E. Puyol-Antón, A. Carass, A. King
In many low-to-middle income (LMIC) countries, ultrasound is used for assessment of pleural effusion. Typically, the extent of the effusion is manually measured by a sonographer, leading to significant intra-/inter-observer variability. In this work, we investigate the use of deep learning (DL) to automate the process of pleural effusion segmentation from ultrasound images. On two datasets acquired in a LMIC setting, we achieve median Dice Similarity Coefficients (DSCs) of 0.82 and 0.74 respectively using the nnU-net DL model. We also investigate the use of coordinate convolutions in the DL model and find that this results in a statistically significant improvement in the median DSC on the first dataset to 0.85, with no significant change on the second dataset. This work showcases, for the first time, the potential of DL in automating the process of effusion assessment from ultrasound in LMIC settings where there is often a lack of experienced radiologists to perform such tasks.
{"title":"Deep Learning-based Segmentation of Pleural Effusion From Ultrasound Using Coordinate Convolutions","authors":"Germain Morilhat, Naomi Kifle, Sandy FinesilverSmith, B. Ruijsink, V. Vergani, Habtamu Tegegne Desita, Z. Desita, E. Puyol-Antón, A. Carass, A. King","doi":"10.48550/arXiv.2208.03305","DOIUrl":"https://doi.org/10.48550/arXiv.2208.03305","url":null,"abstract":"In many low-to-middle income (LMIC) countries, ultrasound is used for assessment of pleural effusion. Typically, the extent of the effusion is manually measured by a sonographer, leading to significant intra-/inter-observer variability. In this work, we investigate the use of deep learning (DL) to automate the process of pleural effusion segmentation from ultrasound images. On two datasets acquired in a LMIC setting, we achieve median Dice Similarity Coefficients (DSCs) of 0.82 and 0.74 respectively using the nnU-net DL model. We also investigate the use of coordinate convolutions in the DL model and find that this results in a statistically significant improvement in the median DSC on the first dataset to 0.85, with no significant change on the second dataset. This work showcases, for the first time, the potential of DL in automating the process of effusion assessment from ultrasound in LMIC settings where there is often a lack of experienced radiologists to perform such tasks.","PeriodicalId":347091,"journal":{"name":"DeCaF/FAIR@MICCAI","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130247455","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-29DOI: 10.48550/arXiv.2207.14625
Malte Tölle, U. Köthe, F. André, B. Meder, S. Engelhardt
Differential privacy (DP) has arisen as the gold standard in protecting an individual's privacy in datasets by adding calibrated noise to each data sample. While the application to categorical data is straightforward, its usability in the context of images has been limited. Contrary to categorical data the meaning of an image is inherent in the spatial correlation of neighboring pixels making the simple application of noise infeasible. Invertible Neural Networks (INN) have shown excellent generative performance while still providing the ability to quantify the exact likelihood. Their principle is based on transforming a complicated distribution into a simple one e.g. an image into a spherical Gaussian. We hypothesize that adding noise to the latent space of an INN can enable differentially private image modification. Manipulation of the latent space leads to a modified image while preserving important details. Further, by conditioning the INN on meta-data provided with the dataset we aim at leaving dimensions important for downstream tasks like classification untouched while altering other parts that potentially contain identifying information. We term our method content-aware differential privacy (CADP). We conduct experiments on publicly available benchmarking datasets as well as dedicated medical ones. In addition, we show the generalizability of our method to categorical data. The source code is publicly available at https://github.com/Cardio-AI/CADP.
{"title":"Content-Aware Differential Privacy with Conditional Invertible Neural Networks","authors":"Malte Tölle, U. Köthe, F. André, B. Meder, S. Engelhardt","doi":"10.48550/arXiv.2207.14625","DOIUrl":"https://doi.org/10.48550/arXiv.2207.14625","url":null,"abstract":"Differential privacy (DP) has arisen as the gold standard in protecting an individual's privacy in datasets by adding calibrated noise to each data sample. While the application to categorical data is straightforward, its usability in the context of images has been limited. Contrary to categorical data the meaning of an image is inherent in the spatial correlation of neighboring pixels making the simple application of noise infeasible. Invertible Neural Networks (INN) have shown excellent generative performance while still providing the ability to quantify the exact likelihood. Their principle is based on transforming a complicated distribution into a simple one e.g. an image into a spherical Gaussian. We hypothesize that adding noise to the latent space of an INN can enable differentially private image modification. Manipulation of the latent space leads to a modified image while preserving important details. Further, by conditioning the INN on meta-data provided with the dataset we aim at leaving dimensions important for downstream tasks like classification untouched while altering other parts that potentially contain identifying information. We term our method content-aware differential privacy (CADP). We conduct experiments on publicly available benchmarking datasets as well as dedicated medical ones. In addition, we show the generalizability of our method to categorical data. The source code is publicly available at https://github.com/Cardio-AI/CADP.","PeriodicalId":347091,"journal":{"name":"DeCaF/FAIR@MICCAI","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122009089","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-05-05DOI: 10.48550/arXiv.2205.02652
Dmitrii Usynin, Helena Klause, D. Rueckert, Georgios Kaissis
In federated learning for medical image analysis, the safety of the learning protocol is paramount. Such settings can often be compromised by adversaries that target either the private data used by the federation or the integrity of the model itself. This requires the medical imaging community to develop mechanisms to train collaborative models that are private and robust against adversarial data. In response to these challenges, we propose a practical open-source framework to study the effectiveness of combining differential privacy, model compression and adversarial training to improve the robustness of models against adversarial samples under train- and inference-time attacks. Using our framework, we achieve competitive model performance, a significant reduction in model's size and an improved empirical adversarial robustness without a severe performance degradation, critical in medical image analysis.
{"title":"Can collaborative learning be private, robust and scalable?","authors":"Dmitrii Usynin, Helena Klause, D. Rueckert, Georgios Kaissis","doi":"10.48550/arXiv.2205.02652","DOIUrl":"https://doi.org/10.48550/arXiv.2205.02652","url":null,"abstract":"In federated learning for medical image analysis, the safety of the learning protocol is paramount. Such settings can often be compromised by adversaries that target either the private data used by the federation or the integrity of the model itself. This requires the medical imaging community to develop mechanisms to train collaborative models that are private and robust against adversarial data. In response to these challenges, we propose a practical open-source framework to study the effectiveness of combining differential privacy, model compression and adversarial training to improve the robustness of models against adversarial samples under train- and inference-time attacks. Using our framework, we achieve competitive model performance, a significant reduction in model's size and an improved empirical adversarial robustness without a severe performance degradation, critical in medical image analysis.","PeriodicalId":347091,"journal":{"name":"DeCaF/FAIR@MICCAI","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-05-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129728337","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1900-01-01DOI: 10.1007/978-3-031-18523-6_2
Yousef Yeganeh, Azade Farshad, Johannes Boschmann, Richard Gaus, Maximilian Frantzen, N. Navab
{"title":"FedAP: Adaptive Personalization in Federated Learning for Non-IID Data","authors":"Yousef Yeganeh, Azade Farshad, Johannes Boschmann, Richard Gaus, Maximilian Frantzen, N. Navab","doi":"10.1007/978-3-031-18523-6_2","DOIUrl":"https://doi.org/10.1007/978-3-031-18523-6_2","url":null,"abstract":"","PeriodicalId":347091,"journal":{"name":"DeCaF/FAIR@MICCAI","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122608810","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1900-01-01DOI: 10.1007/978-3-031-18523-6_15
Kaveri A. Thakoor, Ari Carter, Ge Song, Adam Wax, Omar Moussa, Royce Chen, C. Hendon, P. Sajda
{"title":"Enhancing Portable OCT Image Quality via GANs for AI-Based Eye Disease Detection","authors":"Kaveri A. Thakoor, Ari Carter, Ge Song, Adam Wax, Omar Moussa, Royce Chen, C. Hendon, P. Sajda","doi":"10.1007/978-3-031-18523-6_15","DOIUrl":"https://doi.org/10.1007/978-3-031-18523-6_15","url":null,"abstract":"","PeriodicalId":347091,"journal":{"name":"DeCaF/FAIR@MICCAI","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125721907","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1900-01-01DOI: 10.1007/978-3-031-18523-6_10
Garima Aggarwal, Chun-Yin Huang, Dian Fan, Xiaoxiao Li, Zehua Wang
{"title":"DeMed: A Novel and Efficient Decentralized Learning Framework for Medical Images Classification on Blockchain","authors":"Garima Aggarwal, Chun-Yin Huang, Dian Fan, Xiaoxiao Li, Zehua Wang","doi":"10.1007/978-3-031-18523-6_10","DOIUrl":"https://doi.org/10.1007/978-3-031-18523-6_10","url":null,"abstract":"","PeriodicalId":347091,"journal":{"name":"DeCaF/FAIR@MICCAI","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127438305","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：