Pub Date : 2024-07-06DOI: 10.1007/s12559-024-10319-8
Mengke Liu, Wenbing Zhang, Guanglei Wu
This article examines the prescribed-time sampled-data control problem for multi-agent systems in signed networks. A time-varying high gain-based protocol is devised to solve the prescribed-time bipartite consensus problem of the linear multi-agent systems with the control gain matrix being resolved through the utilization of the parametric Lyapunov equation. By using the method of scalarization, sufficient conditions are attained to ensure the prescribed-time bipartite consensus of linear multi-agent systems, where the maximum allowable sampling interval (MASI) ensuring the prescribed-time consensus is determined by the initial values of the system state, the linear dynamics of the system, and the maximum eigenvalue of the Laplacian matrix. Specifically, the MASI is inversely proportional to the maximum eigenvalue of the Laplacian matrix. Finally, the validity of the conclusion is ensured through numerical simulation.
{"title":"Prescribed-Time Sampled-Data Control for the Bipartite Consensus of Linear Multi-Agent Systems in Singed Networks","authors":"Mengke Liu, Wenbing Zhang, Guanglei Wu","doi":"10.1007/s12559-024-10319-8","DOIUrl":"https://doi.org/10.1007/s12559-024-10319-8","url":null,"abstract":"<p>This article examines the prescribed-time sampled-data control problem for multi-agent systems in signed networks. A time-varying high gain-based protocol is devised to solve the prescribed-time bipartite consensus problem of the linear multi-agent systems with the control gain matrix being resolved through the utilization of the parametric Lyapunov equation. By using the method of scalarization, sufficient conditions are attained to ensure the prescribed-time bipartite consensus of linear multi-agent systems, where the maximum allowable sampling interval (MASI) ensuring the prescribed-time consensus is determined by the initial values of the system state, the linear dynamics of the system, and the maximum eigenvalue of the Laplacian matrix. Specifically, the MASI is inversely proportional to the maximum eigenvalue of the Laplacian matrix. Finally, the validity of the conclusion is ensured through numerical simulation.</p>","PeriodicalId":51243,"journal":{"name":"Cognitive Computation","volume":"2016 1","pages":""},"PeriodicalIF":5.4,"publicationDate":"2024-07-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141572134","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-06DOI: 10.1007/s12559-024-10307-y
Huiru Wang, Xiaoqing Hong, Siyuan Zhang
In order to obtain a more robust and sparse classifier, in this paper, we propose a novel classifier termed as twin bounded support vector machine with capped pinball loss (CPin-TBSVM), which has the excellent properties of being insensitive to feature and label noise. Given that the proposed model is non-convex, we use the convex-concave procedure algorithm (CCCP) to solve a series of two smaller-sized quadratic programming problems to find the optimal solution. In the process of solving the iterative subproblem, the dual coordinate descent method (DCDM) is used for speeding up solving optimization problems. Moreover, we analyze its theoretical properties, including that the capped pinball loss satisfies Bayes’ rule and CPin-TBSVM has certain noise insensitivity and sparsity. The properties are verified on an artificial dataset as well. The numerical experiment is conducted on 24 UCI datasets and the results are compared with four other models which include SVM, TSVM, Pin-GTSVM and TPin-TSVM. The results show that the proposed CPin-TBSVM has a better classification effect and noise insensitivity.
{"title":"Twin Bounded Support Vector Machine with Capped Pinball Loss","authors":"Huiru Wang, Xiaoqing Hong, Siyuan Zhang","doi":"10.1007/s12559-024-10307-y","DOIUrl":"https://doi.org/10.1007/s12559-024-10307-y","url":null,"abstract":"<p>In order to obtain a more robust and sparse classifier, in this paper, we propose a novel classifier termed as twin bounded support vector machine with capped pinball loss (CPin-TBSVM), which has the excellent properties of being insensitive to feature and label noise. Given that the proposed model is non-convex, we use the convex-concave procedure algorithm (CCCP) to solve a series of two smaller-sized quadratic programming problems to find the optimal solution. In the process of solving the iterative subproblem, the dual coordinate descent method (DCDM) is used for speeding up solving optimization problems. Moreover, we analyze its theoretical properties, including that the capped pinball loss satisfies Bayes’ rule and CPin-TBSVM has certain noise insensitivity and sparsity. The properties are verified on an artificial dataset as well. The numerical experiment is conducted on 24 UCI datasets and the results are compared with four other models which include SVM, TSVM, Pin-GTSVM and TPin-TSVM. The results show that the proposed CPin-TBSVM has a better classification effect and noise insensitivity.</p>","PeriodicalId":51243,"journal":{"name":"Cognitive Computation","volume":"44 1","pages":""},"PeriodicalIF":5.4,"publicationDate":"2024-07-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141577786","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-05DOI: 10.1007/s12559-024-10313-0
Jihene Tmamna, Emna Ben Ayed, Rahma Fourati, Mandar Gogate, Tughrul Arslan, Amir Hussain, Mounir Ben Ayed
Over the past few years, larger and deeper neural network models, particularly convolutional neural networks (CNNs), have consistently advanced state-of-the-art performance across various disciplines. Yet, the computational demands of these models have escalated exponentially. Intensive computations hinder not only research inclusiveness and deployment on resource-constrained devices, such as Edge Internet of Things (IoT) devices, but also result in a substantial carbon footprint. Green deep learning has emerged as a research field that emphasizes energy consumption and carbon emissions during model training and inference, aiming to innovate with light and energy-efficient neural networks. Various techniques are available to achieve this goal. Studies show that conventional deep models often contain redundant parameters that do not alter outcomes significantly, underpinning the theoretical basis for model pruning. Consequently, this timely review paper seeks to systematically summarize recent breakthroughs in CNN pruning methods, offering necessary background knowledge for researchers in this interdisciplinary domain. Secondly, we spotlight the challenges of current model pruning methods to inform future avenues of research. Additionally, the survey highlights the pressing need for the development of innovative metrics to effectively balance diverse pruning objectives. Lastly, it investigates pruning techniques oriented towards sophisticated deep learning models, including hybrid feedforward CNNs and long short-term memory (LSTM) recurrent neural networks, a field ripe for exploration within green deep learning research.
{"title":"Pruning Deep Neural Networks for Green Energy-Efficient Models: A Survey","authors":"Jihene Tmamna, Emna Ben Ayed, Rahma Fourati, Mandar Gogate, Tughrul Arslan, Amir Hussain, Mounir Ben Ayed","doi":"10.1007/s12559-024-10313-0","DOIUrl":"https://doi.org/10.1007/s12559-024-10313-0","url":null,"abstract":"<p>Over the past few years, larger and deeper neural network models, particularly convolutional neural networks (CNNs), have consistently advanced state-of-the-art performance across various disciplines. Yet, the computational demands of these models have escalated exponentially. Intensive computations hinder not only research inclusiveness and deployment on resource-constrained devices, such as Edge Internet of Things (IoT) devices, but also result in a substantial carbon footprint. Green deep learning has emerged as a research field that emphasizes energy consumption and carbon emissions during model training and inference, aiming to innovate with light and energy-efficient neural networks. Various techniques are available to achieve this goal. Studies show that conventional deep models often contain redundant parameters that do not alter outcomes significantly, underpinning the theoretical basis for model pruning. Consequently, this timely review paper seeks to systematically summarize recent breakthroughs in CNN pruning methods, offering necessary background knowledge for researchers in this interdisciplinary domain. Secondly, we spotlight the challenges of current model pruning methods to inform future avenues of research. Additionally, the survey highlights the pressing need for the development of innovative metrics to effectively balance diverse pruning objectives. Lastly, it investigates pruning techniques oriented towards sophisticated deep learning models, including hybrid feedforward CNNs and long short-term memory (LSTM) recurrent neural networks, a field ripe for exploration within green deep learning research.</p>","PeriodicalId":51243,"journal":{"name":"Cognitive Computation","volume":"67 1","pages":""},"PeriodicalIF":5.4,"publicationDate":"2024-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141547232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-02DOI: 10.1007/s12559-024-10318-9
Rana Alabdan, Jamal Alsamri, Siwar Ben Haj Hassine, Faiz Abdullah Alotaibi, Saud S. Alotaibi, Ayman Yafoz, Mrim M. Alnfiai, Mesfer Al Duhayyim
The emergence of deep learning (DL) has improved the excellence of generated media. However, with the enlarged level of photorealism, synthetic media is becoming very similar to tangible media, increasing severe problems regarding transmitting fake or deployed data over the Internet. In this situation, it is significant to improve automatic tools to constantly and early identify synthetic media. Generative Adversarial Network (GAN)-based models can create realistic faces that cause deep social and security issues. Existing techniques for identifying GAN-generated faces can execute well on restricted public datasets. Nevertheless, images from existing datasets must signify real situations sufficient for view variants and data distributions, where real faces mainly outnumber artificial ones. Therefore, this study develops an optimal DL-based GAN-generated face detection and classification (ODL-GANFDC) technique. The ODL-GANFDC technique aims to examine the input images properly and recognize whether GAN generates them. To accomplish this, the ODL-GANFDC technique follows the initial stage of the CLAHE-based contrast enhancement process. In addition, the deep residual network (DRN) model must be employed to learn the complex and intrinsic patterns from the preprocessed images. Besides, the hyperparameters of the DRN model can be optimally chosen using an improved sand cat swarm optimization (ISCSO) algorithm. Finally, the GAN-generated faces can be detected using a variational autoencoder (VAE). An extensive set of experimentations can be carried out to highlight the performance of the ODL-GANFDC technique. The experimental outcomes stated the promising results of the ODL-GANFDC technique over compared approaches on the GAN-generated face detection process.
深度学习(DL)的出现提高了生成媒体的质量。然而,随着逼真度的提高,合成媒体正变得与有形媒体非常相似,从而增加了在互联网上传输伪造或部署数据的严重问题。在这种情况下,改进自动工具以不断及早识别合成媒体就显得尤为重要。基于生成对抗网络(GAN)的模型可以创建逼真的人脸,从而引发深刻的社会和安全问题。识别 GAN 生成的人脸的现有技术可以在受限的公共数据集上很好地执行。然而,现有数据集中的图像必须足以代表视图变体和数据分布的真实情况,在这种情况下,真实人脸主要多于人造人脸。因此,本研究开发了一种基于 DL 的 GAN 生成的最佳人脸检测和分类(ODL-GANFDC)技术。ODL-GANFDC 技术旨在正确检查输入图像并识别 GAN 是否生成了这些图像。为此,ODL-GANFDC 技术遵循基于 CLAHE 的对比度增强过程的初始阶段。此外,还必须使用深度残差网络(DRN)模型来学习预处理图像中复杂的内在模式。此外,DRN 模型的超参数可通过改进的沙猫群优化(ISCSO)算法进行优化选择。最后,可以使用变异自动编码器(VAE)检测 GAN 生成的人脸。为了突出 ODL-GANFDC 技术的性能,我们进行了大量实验。实验结果表明,在 GAN 生成的人脸检测过程中,ODL-GANFDC 技术与其他方法相比具有良好的效果。
{"title":"Unmasking GAN-Generated Faces with Optimal Deep Learning and Cognitive Computing-Based Cutting-Edge Detection System","authors":"Rana Alabdan, Jamal Alsamri, Siwar Ben Haj Hassine, Faiz Abdullah Alotaibi, Saud S. Alotaibi, Ayman Yafoz, Mrim M. Alnfiai, Mesfer Al Duhayyim","doi":"10.1007/s12559-024-10318-9","DOIUrl":"https://doi.org/10.1007/s12559-024-10318-9","url":null,"abstract":"<p>The emergence of deep learning (DL) has improved the excellence of generated media. However, with the enlarged level of photorealism, synthetic media is becoming very similar to tangible media, increasing severe problems regarding transmitting fake or deployed data over the Internet. In this situation, it is significant to improve automatic tools to constantly and early identify synthetic media. Generative Adversarial Network (GAN)-based models can create realistic faces that cause deep social and security issues. Existing techniques for identifying GAN-generated faces can execute well on restricted public datasets. Nevertheless, images from existing datasets must signify real situations sufficient for view variants and data distributions, where real faces mainly outnumber artificial ones. Therefore, this study develops an optimal DL-based GAN-generated face detection and classification (ODL-GANFDC) technique. The ODL-GANFDC technique aims to examine the input images properly and recognize whether GAN generates them. To accomplish this, the ODL-GANFDC technique follows the initial stage of the CLAHE-based contrast enhancement process. In addition, the deep residual network (DRN) model must be employed to learn the complex and intrinsic patterns from the preprocessed images. Besides, the hyperparameters of the DRN model can be optimally chosen using an improved sand cat swarm optimization (ISCSO) algorithm. Finally, the GAN-generated faces can be detected using a variational autoencoder (VAE). An extensive set of experimentations can be carried out to highlight the performance of the ODL-GANFDC technique. The experimental outcomes stated the promising results of the ODL-GANFDC technique over compared approaches on the GAN-generated face detection process.\u0000</p>","PeriodicalId":51243,"journal":{"name":"Cognitive Computation","volume":"8 1","pages":""},"PeriodicalIF":5.4,"publicationDate":"2024-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141502718","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In contemporary real-time applications, diminutive devices are increasingly employing a greater portion of the spectrum to transmit data despite the relatively small size of said data. The demand for big data in cloud storage networks is on the rise, as cognitive networks can enable intelligent decision-making with minimal spectrum utilization. The introduction of cognitive networks has facilitated the provision of a novel channel that enables the allocation of low power resources while minimizing path loss. The proposed method involves the integration of three algorithms to examine the process of big data. Whenever big data applications are examined then distance measurement, decisions mechanism and learning techniques from past data is much importance thus algorithms are chosen according to the requirements of big data and cloud storage networks. Further the effect of integration process is examined with three case studies that considers low resource, path loss and weight functions where optimized outcome is achieved in all defined case studies as compared to existing approach.
{"title":"Cognitive Intelligent Decisions for Big Data and Cloud Computing in Industrial Applications using Trifold Algorithms","authors":"Shitharth Selvarajan, Hariprasath Manoharan, Rakan A. Alsowail, Achyut Shankar, Saravanan Pandiaraj, Carsten Maple, Wattana Viriyasitavat","doi":"10.1007/s12559-024-10317-w","DOIUrl":"https://doi.org/10.1007/s12559-024-10317-w","url":null,"abstract":"<p>In contemporary real-time applications, diminutive devices are increasingly employing a greater portion of the spectrum to transmit data despite the relatively small size of said data. The demand for big data in cloud storage networks is on the rise, as cognitive networks can enable intelligent decision-making with minimal spectrum utilization. The introduction of cognitive networks has facilitated the provision of a novel channel that enables the allocation of low power resources while minimizing path loss. The proposed method involves the integration of three algorithms to examine the process of big data. Whenever big data applications are examined then distance measurement, decisions mechanism and learning techniques from past data is much importance thus algorithms are chosen according to the requirements of big data and cloud storage networks. Further the effect of integration process is examined with three case studies that considers low resource, path loss and weight functions where optimized outcome is achieved in all defined case studies as compared to existing approach.\u0000</p>","PeriodicalId":51243,"journal":{"name":"Cognitive Computation","volume":"46 1","pages":""},"PeriodicalIF":5.4,"publicationDate":"2024-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141502779","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In recent years, automatic disease diagnosis has gained immense popularity in research and industry communities. Humans learn a task through both successful and unsuccessful attempts in real life, and physicians are not different. When doctors fail to diagnose disease correctly, they re-assess the extracted symptoms and re-diagnose the patient by inspecting a few more symptoms guided by their previous experience and current context. Motivated by the experience gained from failure assessment, we propose a novel end-to-end automatic disease diagnosis dialogue system called Failure Assessment incorporated Symptom Investigation and Disease Diagnosis (FA-SIDD) Assistant. The proposed FA-SIDD model includes a knowledge-guided, incorrect disease projection-aware failure assessment module that analyzes unsuccessful diagnosis attempts and reinforces the assessment for further investigation and re-diagnosis. We formulate a novel Markov decision process for the proposed failure assessment, incorporating symptom investigation and disease diagnosis frameworks, and optimize the policy using deep reinforcement learning. The proposed model has outperformed several baselines and the existing symptom investigation and diagnosis methods by a significant margin (1–3%) in all evaluation metrics (including human evaluation). The improvements over the multiple datasets and across multiple algorithms firmly establish the efficacy of learning gained from unsuccessful diagnoses. The work is the first attempt that investigate the importance of learning gained from unsuccessful diagnoses. The developed assistant learns diagnosis task more efficiently than traditional assistants and shows robust behavior. Furthermore, the code is available at https://github.com/AbhisekTiwari/FA-SIDA.
{"title":"Learning from Failure: Towards Developing a Disease Diagnosis Assistant That Also Learns from Unsuccessful Diagnoses","authors":"Abhisek Tiwari, Swarna S, Sriparna Saha, Pushpak Bhattacharyya, Minakshi Dhar, Sarbajeet Tiwari","doi":"10.1007/s12559-024-10274-4","DOIUrl":"https://doi.org/10.1007/s12559-024-10274-4","url":null,"abstract":"<p>In recent years, automatic disease diagnosis has gained immense popularity in research and industry communities. Humans learn a task through both successful and unsuccessful attempts in real life, and physicians are not different. When doctors fail to diagnose disease correctly, they re-assess the extracted symptoms and re-diagnose the patient by inspecting a few more symptoms guided by their previous experience and current context. Motivated by the experience gained from failure assessment, we propose a novel end-to-end automatic disease diagnosis dialogue system called Failure Assessment incorporated Symptom Investigation and Disease Diagnosis (FA-SIDD) Assistant. The proposed FA-SIDD model includes a knowledge-guided, incorrect disease projection-aware failure assessment module that analyzes unsuccessful diagnosis attempts and reinforces the assessment for further investigation and re-diagnosis. We formulate a novel Markov decision process for the proposed failure assessment, incorporating symptom investigation and disease diagnosis frameworks, and optimize the policy using deep reinforcement learning. The proposed model has outperformed several baselines and the existing symptom investigation and diagnosis methods by a significant margin (1–3%) in all evaluation metrics (including human evaluation). The improvements over the multiple datasets and across multiple algorithms firmly establish the efficacy of learning gained from unsuccessful diagnoses. The work is the first attempt that investigate the importance of learning gained from unsuccessful diagnoses. The developed assistant learns diagnosis task more efficiently than traditional assistants and shows robust behavior. Furthermore, the code is available at https://github.com/AbhisekTiwari/FA-SIDA.</p>","PeriodicalId":51243,"journal":{"name":"Cognitive Computation","volume":"49 1","pages":""},"PeriodicalIF":5.4,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141502721","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
With the widespread adoption of deep learning, there has been a notable increase in the prevalence of multimodal deepfake content. These deepfakes pose a substantial risk to both individual privacy and the security of their assets. In response to this pressing issue, researchers have undertaken substantial endeavors in utilizing generative AI and cognitive computation to leverage multimodal data to detect deepfakes. However, the efforts thus far have fallen short of fully exploiting the extensive reservoir of multimodal feature information, which leads to a deficiency in leveraging spatial information across multiple dimensions. In this study, we introduce a framework called Visual-Language Pretraining with Gate Fusion (VLP-GF), designed to identify multimodal deceptive content and enhance the accurate localization of manipulated regions within both images and textual annotations. Specifically, we introduce an adaptive fusion module tailored to integrate local and global information simultaneously. This module captures global context and local details concurrently, thereby improving the performance of image bounding-box grounding within the system. Additionally, to maximize the utilization of semantic information from diverse modalities, we incorporate a gating mechanism to strengthen the interaction of multimodal information further. Through a series of ablation experiments and comprehensive comparisons with state-of-the-art approaches on extensive benchmark datasets, we empirically demonstrate the superior efficacy of VLP-GF.
{"title":"Multi-Modal Generative DeepFake Detection via Visual-Language Pretraining with Gate Fusion for Cognitive Computation","authors":"Guisheng Zhang, Mingliang Gao, Qilei Li, Wenzhe Zhai, Gwanggil Jeon","doi":"10.1007/s12559-024-10316-x","DOIUrl":"https://doi.org/10.1007/s12559-024-10316-x","url":null,"abstract":"<p>With the widespread adoption of deep learning, there has been a notable increase in the prevalence of multimodal deepfake content. These deepfakes pose a substantial risk to both individual privacy and the security of their assets. In response to this pressing issue, researchers have undertaken substantial endeavors in utilizing generative AI and cognitive computation to leverage multimodal data to detect deepfakes. However, the efforts thus far have fallen short of fully exploiting the extensive reservoir of multimodal feature information, which leads to a deficiency in leveraging spatial information across multiple dimensions. In this study, we introduce a framework called Visual-Language Pretraining with Gate Fusion (VLP-GF), designed to identify multimodal deceptive content and enhance the accurate localization of manipulated regions within both images and textual annotations. Specifically, we introduce an adaptive fusion module tailored to integrate local and global information simultaneously. This module captures global context and local details concurrently, thereby improving the performance of image bounding-box grounding within the system. Additionally, to maximize the utilization of semantic information from diverse modalities, we incorporate a gating mechanism to strengthen the interaction of multimodal information further. Through a series of ablation experiments and comprehensive comparisons with state-of-the-art approaches on extensive benchmark datasets, we empirically demonstrate the superior efficacy of VLP-GF.</p>","PeriodicalId":51243,"journal":{"name":"Cognitive Computation","volume":"145 1","pages":""},"PeriodicalIF":5.4,"publicationDate":"2024-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141502719","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-21DOI: 10.1007/s12559-024-10314-z
Ziqin Zhao, Fan Lyu, Linyan Li, Fuyuan Hu, Minming Gu, Li Sun
Background
Federated Continual Learning (FCL) involves learning from distributed data on edge devices with incremental knowledge. However, current FCL methods struggle to retain long-term memories on the server.
Method
In this paper, we introduce a method called Fisher INformation Accumulation Learning (FINAL) to address catastrophic forgetting in FCL. First, we accumulate a global Fisher with a federated Fisher information matrix formed from clients task by task to remember long-term knowledge. Second, we present a novel multi-node collaborative integration strategy to assemble the federated Fisher, which reveals the task-specific co-importance of parameters among clients. Finally, we raise a Fisher balancing method to combine the global Fisher and federated Fisher, avoiding neglecting new learning or causing catastrophic forgetting.
Results
We conducted evaluations on four FCL datasets, and the findings demonstrate that the proposed FINAL effectively maintains long-term knowledge on the server.
Conclusions
The exceptional performance of this method indicates its significant value for future FCL research.
背景联合持续学习(FCL)涉及从边缘设备上的分布式数据中学习增量知识。方法在本文中,我们介绍了一种名为费舍尔信息积累学习(FINAL)的方法,以解决 FCL 中的灾难性遗忘问题。首先,我们用一个由客户逐个任务形成的联合 Fisher 信息矩阵来积累全局 Fisher,从而记住长期知识。其次,我们提出了一种新颖的多节点协作集成策略来组装联合费雪,从而揭示了客户间特定任务参数的共同重要性。最后,我们提出了一种费舍尔平衡方法,将全局费舍尔和联合费舍尔结合起来,避免忽略新的学习或造成灾难性遗忘。结果我们在四个 FCL 数据集上进行了评估,结果表明所提出的 FINAL 有效地维护了服务器上的长期知识。
{"title":"Towards Long-Term Remembering in Federated Continual Learning","authors":"Ziqin Zhao, Fan Lyu, Linyan Li, Fuyuan Hu, Minming Gu, Li Sun","doi":"10.1007/s12559-024-10314-z","DOIUrl":"https://doi.org/10.1007/s12559-024-10314-z","url":null,"abstract":"<h3 data-test=\"abstract-sub-heading\">Background</h3><p>Federated Continual Learning (FCL) involves learning from distributed data on edge devices with incremental knowledge. However, current FCL methods struggle to retain long-term memories on the server.</p><h3 data-test=\"abstract-sub-heading\">Method</h3><p>In this paper, we introduce a method called Fisher INformation Accumulation Learning (FINAL) to address catastrophic forgetting in FCL. First, we accumulate a global Fisher with a federated Fisher information matrix formed from clients task by task to remember long-term knowledge. Second, we present a novel multi-node collaborative integration strategy to assemble the federated Fisher, which reveals the task-specific co-importance of parameters among clients. Finally, we raise a Fisher balancing method to combine the global Fisher and federated Fisher, avoiding neglecting new learning or causing catastrophic forgetting.</p><h3 data-test=\"abstract-sub-heading\">Results</h3><p>We conducted evaluations on four FCL datasets, and the findings demonstrate that the proposed FINAL effectively maintains long-term knowledge on the server.</p><h3 data-test=\"abstract-sub-heading\">Conclusions</h3><p>The exceptional performance of this method indicates its significant value for future FCL research.</p>","PeriodicalId":51243,"journal":{"name":"Cognitive Computation","volume":"86 1","pages":""},"PeriodicalIF":5.4,"publicationDate":"2024-06-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141502734","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-17DOI: 10.1007/s12559-024-10310-3
Mahmuda Akter, Nour Moustafa, Benjamin Turnbull
Smart healthcare systems promise significant benefits for fast and accurate medical decisions. However, working with personal health data presents new privacy issues and constraints that must be solved from a cybersecurity perspective. Edge intelligence-enabled federated learning is a new scheme that utilises decentralised computing that allows data analytics to be carried out at the edge of a network, enhancing data privacy. However, this scheme suffers from privacy attacks, including inference, free-riding, and man-in-the-middle attacks, especially with serverless computing for allocating resources to user needs. Edge intelligence-enabled federated learning requires client data insertion and deletion to authenticate genuine clients and a serverless computing capability to ensure the security of collaborative machine learning models. This work introduces a serverless privacy edge intelligence-based federated learning (SPEI-FL) framework to address these issues. SPEI-FL includes a federated edge aggregator and authentication method to improve the data privacy of federated learning and allow client adaptation and removal without impacting the overall learning processes. It also can classify intruders through serverless computing processes. The proposed framework was evaluated with the unstructured COVID-19 medical chest x-rays and MNIST digit datasets, and the structured BoT-IoT dataset. The performance of the framework is comparable with existing authentication methods and reported a higher accuracy than comparable methods (approximately 90% as compared with the 81% reported by peer methods). The proposed authentication method prevents the exposure of sensitive patient information during medical device authentication and would become the cornerstone of the next generation of medical security with serverless computing.
{"title":"SPEI-FL: Serverless Privacy Edge Intelligence-Enabled Federated Learning in Smart Healthcare Systems","authors":"Mahmuda Akter, Nour Moustafa, Benjamin Turnbull","doi":"10.1007/s12559-024-10310-3","DOIUrl":"https://doi.org/10.1007/s12559-024-10310-3","url":null,"abstract":"<p>Smart healthcare systems promise significant benefits for fast and accurate medical decisions. However, working with personal health data presents new privacy issues and constraints that must be solved from a cybersecurity perspective. Edge intelligence-enabled federated learning is a new scheme that utilises decentralised computing that allows data analytics to be carried out at the edge of a network, enhancing data privacy. However, this scheme suffers from privacy attacks, including inference, free-riding, and man-in-the-middle attacks, especially with serverless computing for allocating resources to user needs. Edge intelligence-enabled federated learning requires client data insertion and deletion to authenticate genuine clients and a serverless computing capability to ensure the security of collaborative machine learning models. This work introduces a serverless privacy edge intelligence-based federated learning (SPEI-FL) framework to address these issues. SPEI-FL includes a federated edge aggregator and authentication method to improve the data privacy of federated learning and allow client adaptation and removal without impacting the overall learning processes. It also can classify intruders through serverless computing processes. The proposed framework was evaluated with the unstructured COVID-19 medical chest x-rays and MNIST digit datasets, and the structured BoT-IoT dataset. The performance of the framework is comparable with existing authentication methods and reported a higher accuracy than comparable methods (approximately 90% as compared with the 81% reported by peer methods). The proposed authentication method prevents the exposure of sensitive patient information during medical device authentication and would become the cornerstone of the next generation of medical security with serverless computing.</p>","PeriodicalId":51243,"journal":{"name":"Cognitive Computation","volume":"86 1","pages":""},"PeriodicalIF":5.4,"publicationDate":"2024-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141502720","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-06-14DOI: 10.1007/s12559-024-10315-y
Zhirui Zeng, Jialing He, Tao Xiang, Ning Wang, Biwen Chen, Shangwei Guo
The burgeoning practice of unauthorized acquisition and utilization of personal textual data (e.g., social media comments and search histories) by certain entities has become a discernible trend. To uphold data protection regulations such as the Asia–Pacific Privacy Initiative (APPI) and to identify instances of unpermitted exploitation of personal data, we propose a novel and efficient audit framework that helps users conduct cognitive analysis to determine if their textual data was used for data augmentation. and training a discriminative model. In particular, we focus on auditing models that use BERT as the backbone for discriminating text and are at the core of popular online services. We first propose an accumulated discrepancy score, which involves not only the response of the target model to the auditing sample but also the responses between pre-trained and finetuned models, to identify membership. We implement two types of audit methods (i.e., sample-level and user-level) according to our framework and conduct comprehensive experiments on two downstream applications to evaluate the performance. The experimental results demonstrate that our sample-level auditing achieves an AUC of 89.7% and an accuracy of 83%, whereas the user-level method can audit membership with an AUC of 89.7% and an accuracy of 88%. Additionally, we undertake an analysis of how augmentation methods impact auditing performance and expound upon the underlying reasons for these observations.
{"title":"Cognitive Tracing Data Trails: Auditing Data Provenance in Discriminative Language Models Using Accumulated Discrepancy Score","authors":"Zhirui Zeng, Jialing He, Tao Xiang, Ning Wang, Biwen Chen, Shangwei Guo","doi":"10.1007/s12559-024-10315-y","DOIUrl":"https://doi.org/10.1007/s12559-024-10315-y","url":null,"abstract":"<p>The burgeoning practice of unauthorized acquisition and utilization of personal textual data (e.g., social media comments and search histories) by certain entities has become a discernible trend. To uphold data protection regulations such as the Asia–Pacific Privacy Initiative (APPI) and to identify instances of unpermitted exploitation of personal data, we propose a novel and efficient audit framework that helps users conduct cognitive analysis to determine if their textual data was used for data augmentation. and training a discriminative model. In particular, we focus on auditing models that use BERT as the backbone for discriminating text and are at the core of popular online services. We first propose an accumulated discrepancy score, which involves not only the response of the target model to the auditing sample but also the responses between pre-trained and finetuned models, to identify membership. We implement two types of audit methods (i.e., sample-level and user-level) according to our framework and conduct comprehensive experiments on two downstream applications to evaluate the performance. The experimental results demonstrate that our sample-level auditing achieves an AUC of 89.7% and an accuracy of 83%, whereas the user-level method can audit membership with an AUC of 89.7% and an accuracy of 88%. Additionally, we undertake an analysis of how augmentation methods impact auditing performance and expound upon the underlying reasons for these observations.</p>","PeriodicalId":51243,"journal":{"name":"Cognitive Computation","volume":"1 1","pages":""},"PeriodicalIF":5.4,"publicationDate":"2024-06-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141512680","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}