首页 > 最新文献

Future Generation Computer Systems-The International Journal of Escience最新文献

英文 中文
Underwater Mediterranean image analysis based on the compute continuum paradigm 基于连续计算范式的水下地中海图像分析
IF 6.2 2区 计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS Pub Date : 2024-08-12 DOI: 10.1016/j.future.2024.107481

Human activity depends on the oceans for food, transportation, leisure, and many more purposes. Oceans cover 70% of the Earth’s surface, but most of them are unknown to humankind. This is the reason why underwater imaging is a valuable resource asset to Marine Science. Images are acquired with observing systems, e.g. autonomous underwater vehicles or underwater observatories, that presently transmit all the raw data to land stations. However, the transfer of such an amount of data could be challenging, considering the limited power supply and transmission bandwidth of these systems. In this paper, we discuss these aspects, and in particular how it is possible to couple Edge and Cloud computing for effective management of the full processing pipeline according to the Compute Continuum paradigm.

人类活动的食物、交通、休闲和其他许多用途都依赖于海洋。海洋覆盖了地球表面的 70%,但其中大部分不为人类所知。这就是水下成像成为海洋科学宝贵资源的原因。目前,通过观测系统(如自动水下航行器或水下观测站)获取的图像会将所有原始数据传送到陆地观测站。然而,考虑到这些系统的电力供应和传输带宽有限,传输如此大量的数据可能具有挑战性。在本文中,我们将讨论这些方面,特别是如何将边缘计算和云计算结合起来,根据计算连续性范式有效管理整个处理管道。
{"title":"Underwater Mediterranean image analysis based on the compute continuum paradigm","authors":"","doi":"10.1016/j.future.2024.107481","DOIUrl":"10.1016/j.future.2024.107481","url":null,"abstract":"<div><p>Human activity depends on the oceans for food, transportation, leisure, and many more purposes. Oceans cover 70% of the Earth’s surface, but most of them are unknown to humankind. This is the reason why underwater imaging is a valuable resource asset to Marine Science. Images are acquired with observing systems, e.g. autonomous underwater vehicles or underwater observatories, that presently transmit all the raw data to land stations. However, the transfer of such an amount of data could be challenging, considering the limited power supply and transmission bandwidth of these systems. In this paper, we discuss these aspects, and in particular how it is possible to couple Edge and Cloud computing for effective management of the full processing pipeline according to the Compute Continuum paradigm.</p></div>","PeriodicalId":55132,"journal":{"name":"Future Generation Computer Systems-The International Journal of Escience","volume":null,"pages":null},"PeriodicalIF":6.2,"publicationDate":"2024-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0167739X2400431X/pdfft?md5=604d28ee54ea8468beac4eeba5484fd0&pid=1-s2.0-S0167739X2400431X-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142020484","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Certificateless Proxy Re-encryption with Cryptographic Reverse Firewalls for Secure Cloud Data Sharing 利用加密反向防火墙进行无证书代理再加密,实现安全的云数据共享
IF 6.2 2区 计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS Pub Date : 2024-08-10 DOI: 10.1016/j.future.2024.08.002

Cloud computing has enabled data-sharing to be more convenient than ever before. However, data security is a major concern that prevents cloud computing from being widely adopted. A potential solution to secure data-sharing in cloud computing is proxy re-encryption (PRE), which allows a proxy to transform encrypted data from one key to another without accessing the plaintext. When using PRE, various challenges arise, including the leak of information by a trusted third party, collusion attacks, and issues associated with revocation. To overcome these challenges, this paper proposes a novel Certificateless Proxy Reencryption with Cryptographic Reverse Firewall for Secure Cloud Data Sharing (CLPRE-CRF). The new scheme enables secure distribution of encrypted data from a data owner to users through public clouds. Meanwhile, the CLPRE-CRF scheme can resist exfiltration of secret information and forgery of ciphertext in case the scheme is compromised. In addition, the scheme provides a flexible revocation mechanism to prevent unauthorized access to private data. The security analysis demonstrates that the CLPRE-CRF resists chosen-plaintext attacks and collusion attacks. Moreover, performance evaluation indicates that our scheme achieves a 14% and 22% reduction in computation costs during the encryption and decryption algorithms, respectively. Therefore, the proposed CLPRE-CRF scheme is well-suited for cloud computing environments.

云计算使数据共享比以往任何时候都更加方便。然而,数据安全是阻碍云计算广泛应用的一个主要问题。云计算中数据安全共享的一个潜在解决方案是代理再加密(PRE),它允许代理将加密数据从一个密钥转换为另一个密钥,而无需访问明文。使用 PRE 时,会出现各种挑战,包括可信第三方泄漏信息、串通攻击以及与撤销相关的问题。为了克服这些挑战,本文提出了一种新型的无证书代理重加密与加密反向防火墙安全云数据共享(CLPRE-CRF)。这一新方案能够通过公共云将加密数据从数据所有者安全地分发到用户。同时,CLPRE-CRF 方案可在方案遭到破坏时抵御秘密信息外泄和密文伪造。此外,该方案还提供了灵活的撤销机制,以防止未经授权访问私人数据。安全性分析表明,CLPRE-CRF 可抵御选择性纯文本攻击和串通攻击。此外,性能评估表明,我们的方案在加密和解密算法中分别减少了 14% 和 22% 的计算成本。因此,所提出的 CLPRE-CRF 方案非常适合云计算环境。
{"title":"Certificateless Proxy Re-encryption with Cryptographic Reverse Firewalls for Secure Cloud Data Sharing","authors":"","doi":"10.1016/j.future.2024.08.002","DOIUrl":"10.1016/j.future.2024.08.002","url":null,"abstract":"<div><p>Cloud computing has enabled data-sharing to be more convenient than ever before. However, data security is a major concern that prevents cloud computing from being widely adopted. A potential solution to secure data-sharing in cloud computing is proxy re-encryption (PRE), which allows a proxy to transform encrypted data from one key to another without accessing the plaintext. When using PRE, various challenges arise, including the leak of information by a trusted third party, collusion attacks, and issues associated with revocation. To overcome these challenges, this paper proposes a novel Certificateless Proxy Reencryption with Cryptographic Reverse Firewall for Secure Cloud Data Sharing (CLPRE-CRF). The new scheme enables secure distribution of encrypted data from a data owner to users through public clouds. Meanwhile, the CLPRE-CRF scheme can resist exfiltration of secret information and forgery of ciphertext in case the scheme is compromised. In addition, the scheme provides a flexible revocation mechanism to prevent unauthorized access to private data. The security analysis demonstrates that the CLPRE-CRF resists chosen-plaintext attacks and collusion attacks. Moreover, performance evaluation indicates that our scheme achieves a 14% and 22% reduction in computation costs during the encryption and decryption algorithms, respectively. Therefore, the proposed CLPRE-CRF scheme is well-suited for cloud computing environments.</p></div>","PeriodicalId":55132,"journal":{"name":"Future Generation Computer Systems-The International Journal of Escience","volume":null,"pages":null},"PeriodicalIF":6.2,"publicationDate":"2024-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141979542","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Network-aware federated neural architecture search 网络感知联合神经架构搜索
IF 6.2 2区 计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS Pub Date : 2024-08-08 DOI: 10.1016/j.future.2024.07.053

The cooperation between Deep Learning (DL) and edge devices has further advanced technological developments, allowing smart devices to serve as both data sources and endpoints for DL-powered applications. However, the success of DL relies on optimal Deep Neural Network (DNN) architectures, and manually developing such systems requires extensive expertise and time. Neural Architecture Search (NAS) has emerged to automate the search for the best-performing neural architectures. Meanwhile, Federated Learning (FL) addresses data privacy concerns by enabling collaborative model development without exchanging the private data of clients.

In a FL system, network limitations can lead to biased model training, slower convergence, and increased communication overhead. On the other hand, traditional DNN architecture design, emphasizing validation accuracy, often overlooks computational efficiency and size constraints of edge devices. This research aims to develop a comprehensive framework that effectively balances trade-offs between model performance, communication efficiency, and the incorporation of FL into an iterative NAS algorithm. This framework aims to overcome challenges by addressing the specific requirements of FL, optimizing DNNs through NAS, and ensuring computational efficiency while considering the network constraints of edge devices.

To address these challenges, we introduce Network-Aware Federated Neural Architecture Search (NAFNAS), an open-source federated neural network pruning framework with network emulation support. Through comprehensive testing, we demonstrate the feasibility of our approach, efficiently reducing DNN size and mitigating communication challenges. Additionally, we propose Network and Distribution Aware Client Grouping (NetDAG), a novel client grouping algorithm tailored for FL with diverse DNN architectures, considerably enhancing efficiency of communication rounds and update balance.

深度学习(DL)与边缘设备之间的合作进一步推动了技术发展,使智能设备既可以作为数据源,也可以作为驱动深度学习应用的终端。然而,深度学习的成功依赖于最佳的深度神经网络(DNN)架构,而手动开发此类系统需要大量的专业知识和时间。神经架构搜索(NAS)的出现可以自动搜索性能最佳的神经架构。同时,联合学习(FL)通过在不交换客户私人数据的情况下实现协作模型开发,解决了数据隐私问题。在联合学习系统中,网络限制可能导致模型训练有偏差、收敛速度变慢以及通信开销增加。另一方面,传统的 DNN 架构设计强调验证准确性,往往忽略了计算效率和边缘设备的尺寸限制。本研究旨在开发一个综合框架,有效平衡模型性能、通信效率和将 FL 纳入迭代 NAS 算法之间的权衡。为了应对这些挑战,我们引入了网络感知联合神经架构搜索(NAFNAS),这是一个支持网络仿真的开源联合神经网络剪枝框架。通过综合测试,我们证明了我们方法的可行性,有效地缩小了 DNN 的规模,缓解了通信挑战。此外,我们还提出了网络和分布感知客户端分组(NetDAG),这是一种新颖的客户端分组算法,专为具有不同 DNN 架构的 FL 量身定制,大大提高了通信轮的效率和更新平衡。
{"title":"Network-aware federated neural architecture search","authors":"","doi":"10.1016/j.future.2024.07.053","DOIUrl":"10.1016/j.future.2024.07.053","url":null,"abstract":"<div><p>The cooperation between Deep Learning (DL) and edge devices has further advanced technological developments, allowing smart devices to serve as both data sources and endpoints for DL-powered applications. However, the success of DL relies on optimal Deep Neural Network (DNN) architectures, and manually developing such systems requires extensive expertise and time. Neural Architecture Search (NAS) has emerged to automate the search for the best-performing neural architectures. Meanwhile, Federated Learning (FL) addresses data privacy concerns by enabling collaborative model development without exchanging the private data of clients.</p><p>In a FL system, network limitations can lead to biased model training, slower convergence, and increased communication overhead. On the other hand, traditional DNN architecture design, emphasizing validation accuracy, often overlooks computational efficiency and size constraints of edge devices. This research aims to develop a comprehensive framework that effectively balances trade-offs between model performance, communication efficiency, and the incorporation of FL into an iterative NAS algorithm. This framework aims to overcome challenges by addressing the specific requirements of FL, optimizing DNNs through NAS, and ensuring computational efficiency while considering the network constraints of edge devices.</p><p>To address these challenges, we introduce Network-Aware Federated Neural Architecture Search (NAFNAS), an open-source federated neural network pruning framework with network emulation support. Through comprehensive testing, we demonstrate the feasibility of our approach, efficiently reducing DNN size and mitigating communication challenges. Additionally, we propose Network and Distribution Aware Client Grouping (NetDAG), a novel client grouping algorithm tailored for FL with diverse DNN architectures, considerably enhancing efficiency of communication rounds and update balance.</p></div>","PeriodicalId":55132,"journal":{"name":"Future Generation Computer Systems-The International Journal of Escience","volume":null,"pages":null},"PeriodicalIF":6.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141992847","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Context aware clustering and meta-heuristic resource allocation for NB-IoT D2D devices in smart healthcare applications 智能医疗保健应用中 NB-IoT D2D 设备的上下文感知聚类和元启发式资源分配
IF 6.2 2区 计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS Pub Date : 2024-08-06 DOI: 10.1016/j.future.2024.08.001

The utilization of Device-to-Device (D2D) communication among Narrowband Internet of Things (NB-IoT) devices offers significant potential for advancing intelligent healthcare systems due to its superior data rates, low power consumption, and spectral efficiency. In D2D communication, strategies to mitigate interference and ensure coexistence with cellular networks are crucial. These strategies are aimed at enhancing user data rates by optimally allocating spectrum and managing the transmission power of D2D devices, presenting a complex engineering challenge. Existing studies are limited either by the inadequate integration of NB-IoT D2D communication methods for healthcare, lacking intelligent, distributed, and autonomous decision-making for reliable data transmission, or by insufficient healthcare event management policies during resource allocation in smart healthcare systems. In this work, we introduce an Intelligent Resource Allocation for Smart Healthcare (iRASH) system, designed to optimize D2D communication within NB-IoT environments. The iRASH innovatively integrates the Density-based Spatial Clustering of Applications with Noise (DBSCAN) and Ant Colony Optimization (ACO) algorithms to effectively address the unique requirements of healthcare applications. The proposed system utilizes Belief-Desire-Intention (BDI) agents for dynamic and intelligent clustering of D2D devices, facilitating autonomous decision-making and efficient resource allocation. This approach not only enhances data transmission rates but also reduces power consumption, and is formulated as a Multi-objective Integer Linear Programming (MILP) problem. Given the NP-hard nature of this problem, iRASH incorporates a polynomial-time meta-heuristic-based ACO algorithm, which provides a suboptimal solution. This algorithm adheres to the principles of distributed D2D communication, promoting equitable resource distribution and substantial improvements in utility, energy efficiency, and scalability. Our system is validated through simulations on the Network Simulator version 3 (NS-3) platform, demonstrating significant advancements over existing state-of-the-art solutions in terms of data rate, power efficiency, and system adaptability. As high as improvements of 35% in utility and 50% in energy cost are demonstrated by the iRASH system compared to the benchmark, proving its effectiveness. The outcomes highlight iRASH’s potential to revolutionize D2D communications in smart healthcare settings, paving the way for more responsive and reliable IoT applications.

窄带物联网(NB-IoT)设备间的设备到设备(D2D)通信具有数据传输速率高、功耗低和频谱效率高等优点,为推进智能医疗系统的发展提供了巨大潜力。在 D2D 通信中,减少干扰并确保与蜂窝网络共存的策略至关重要。这些策略旨在通过优化频谱分配和管理 D2D 设备的传输功率来提高用户数据传输速率,是一项复杂的工程挑战。现有研究受限于 NB-IoT D2D 通信方法在医疗保健领域的不充分集成,缺乏可靠数据传输的智能、分布式和自主决策,或智能医疗保健系统资源分配过程中医疗保健事件管理策略的不足。在这项工作中,我们介绍了智能医疗保健的智能资源分配(iRASH)系统,旨在优化 NB-IoT 环境中的 D2D 通信。iRASH 创新性地集成了基于密度的带噪声应用空间聚类算法(DBSCAN)和蚁群优化算法(ACO),以有效满足医疗保健应用的独特需求。拟议系统利用 "信念-愿望-注意力"(BDI)代理对 D2D 设备进行动态智能聚类,促进自主决策和高效资源分配。这种方法不仅能提高数据传输速率,还能降低功耗,并被表述为一个多目标整数线性规划(MILP)问题。鉴于该问题的 NP 难度,iRASH 采用了基于多项式时间元启发式的 ACO 算法,该算法提供了一个次优解决方案。该算法遵循分布式 D2D 通信原则,促进了资源的公平分配,并大大提高了实用性、能效和可扩展性。我们的系统在网络模拟器第三版(NS-3)平台上进行了仿真验证,在数据传输速率、能效和系统适应性方面都比现有的最先进解决方案有显著进步。与基准相比,iRASH 系统的效用提高了 35%,能源成本降低了 50%,证明了它的有效性。这些成果凸显了 iRASH 在智能医疗保健环境中革新 D2D 通信的潜力,为更灵敏、更可靠的物联网应用铺平了道路。
{"title":"Context aware clustering and meta-heuristic resource allocation for NB-IoT D2D devices in smart healthcare applications","authors":"","doi":"10.1016/j.future.2024.08.001","DOIUrl":"10.1016/j.future.2024.08.001","url":null,"abstract":"<div><p>The utilization of Device-to-Device (D2D) communication among Narrowband Internet of Things (NB-IoT) devices offers significant potential for advancing intelligent healthcare systems due to its superior data rates, low power consumption, and spectral efficiency. In D2D communication, strategies to mitigate interference and ensure coexistence with cellular networks are crucial. These strategies are aimed at enhancing user data rates by optimally allocating spectrum and managing the transmission power of D2D devices, presenting a complex engineering challenge. Existing studies are limited either by the inadequate integration of NB-IoT D2D communication methods for healthcare, lacking intelligent, distributed, and autonomous decision-making for reliable data transmission, or by insufficient healthcare event management policies during resource allocation in smart healthcare systems. In this work, we introduce an Intelligent Resource Allocation for Smart Healthcare (iRASH) system, designed to optimize D2D communication within NB-IoT environments. The iRASH innovatively integrates the Density-based Spatial Clustering of Applications with Noise (DBSCAN) and Ant Colony Optimization (ACO) algorithms to effectively address the unique requirements of healthcare applications. The proposed system utilizes Belief-Desire-Intention (BDI) agents for dynamic and intelligent clustering of D2D devices, facilitating autonomous decision-making and efficient resource allocation. This approach not only enhances data transmission rates but also reduces power consumption, and is formulated as a Multi-objective Integer Linear Programming (MILP) problem. Given the NP-hard nature of this problem, iRASH incorporates a polynomial-time meta-heuristic-based ACO algorithm, which provides a suboptimal solution. This algorithm adheres to the principles of distributed D2D communication, promoting equitable resource distribution and substantial improvements in utility, energy efficiency, and scalability. Our system is validated through simulations on the Network Simulator version 3 (NS-3) platform, demonstrating significant advancements over existing state-of-the-art solutions in terms of data rate, power efficiency, and system adaptability. As high as improvements of 35% in utility and 50% in energy cost are demonstrated by the iRASH system compared to the benchmark, proving its effectiveness. The outcomes highlight iRASH’s potential to revolutionize D2D communications in smart healthcare settings, paving the way for more responsive and reliable IoT applications.</p></div>","PeriodicalId":55132,"journal":{"name":"Future Generation Computer Systems-The International Journal of Escience","volume":null,"pages":null},"PeriodicalIF":6.2,"publicationDate":"2024-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141979541","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Decentralised Identity Management solution for zero-trust multi-domain Computing Continuum frameworks 零信任多域计算的分散式身份管理解决方案 Continuum 框架
IF 6.2 2区 计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS Pub Date : 2024-08-06 DOI: 10.1016/j.future.2024.08.003

The adoption of the Computing Continuum is characterised by the seamless integration of diverse computing environments and devices. In this dynamic landscape, sharing resources across the continuum is becoming a reality and security must move an step forward, specially in terms of authentication and authorisation for such a distributed and heterogeneous environments. The need for robust identity management is paramount and, in this regard, Decentralised Identity Management (DIM) emerges as a promising solution. It leverages decentralised technologies to secure and facilitate identity interactions across the Computing Continuum. Particularly, to enhance security and privacy, it would be desirable to apply the principles of Self-Sovereign Identity (SSI). In this paradigm, users have full ownership and control of their digital identities that empowers individuals to manage and share their identity data on a need-to-know basis. These mechanisms could contribute to improve security properties during continuum resource management operations. In this context, this paper presents the design, workflows and implementation of a solution that provides authentication/authorisation features to distributed zero-trust based infrastructures across the continuum, enhancing security in resource sharing and resource acquisition stages. To this aim, the solution relies on key aspects like decentralisation, interoperability, trust management and privacy-enhancing capabilities. The decentralisation leverages distributed ledger technologies, such as blockchain, to establish a decentralised identity ecosystem. The solution prioritises interoperability, enabling nodes to seamlessly access and share their identities across different domains and environments. Trustworthiness is at the core of DIM, and privacy is also considered, incorporating privacy-preserving techniques that individuals to selectively disclose identity attributes while safeguarding sensitive information. The implementation includes different operations for allowing continuum frameworks to be enhanced with decentralised authentication and authorisation features. The performance has been evaluated measuring the impact for the adoption of the solution. The most expensive task, the self-identity generation, takes only a few seconds (in our deployment) and it is only executed once. Authorisation tasks operate in the millisecond range, which is a totally invaluable time if incorporated into resource acquisition processes in frameworks such as Liqo, used in the scope of FLUIDOS project.

计算连续性的特点是各种计算环境和设备的无缝集成。在这一动态环境中,跨连续体共享资源正在成为现实,安全问题必须向前迈进一步,特别是在这种分布式异构环境的身份验证和授权方面。在这方面,分散式身份管理(DIM)是一个很有前途的解决方案。它利用去中心化技术来确保和促进整个计算过程中的身份互动。特别是,为了提高安全性和隐私性,最好采用自主身份(SSI)原则。在这种模式下,用户对自己的数字身份拥有完全的所有权和控制权,从而使个人有能力在 "需要知道 "的基础上管理和共享自己的身份数据。这些机制有助于提高连续资源管理操作过程中的安全性能。在此背景下,本文介绍了一种解决方案的设计、工作流程和实施,该解决方案可为整个连续体中基于零信任的分布式基础设施提供身份验证/授权功能,从而增强资源共享和资源获取阶段的安全性。为此,该解决方案依赖于去中心化、互操作性、信任管理和隐私增强功能等关键方面。去中心化利用区块链等分布式账本技术,建立一个去中心化的身份生态系统。该解决方案优先考虑互操作性,使节点能够在不同领域和环境中无缝访问和共享其身份。可信性是 DIM 的核心,同时也考虑到了隐私问题,采用了隐私保护技术,让个人在保护敏感信息的同时有选择地披露身份属性。实施过程包括不同的操作,允许连续框架通过分散认证和授权功能得到增强。对性能进行了评估,衡量采用该解决方案的影响。最昂贵的任务--自我身份生成--只需要几秒钟(在我们的部署中),而且只执行一次。授权任务的运行时间仅为几毫秒,如果将其纳入 FLUIDOS 项目所使用的 Liqo 等框架的资源获取流程中,这将是一个非常宝贵的时间。
{"title":"Decentralised Identity Management solution for zero-trust multi-domain Computing Continuum frameworks","authors":"","doi":"10.1016/j.future.2024.08.003","DOIUrl":"10.1016/j.future.2024.08.003","url":null,"abstract":"<div><p>The adoption of the Computing Continuum is characterised by the seamless integration of diverse computing environments and devices. In this dynamic landscape, sharing resources across the continuum is becoming a reality and security must move an step forward, specially in terms of authentication and authorisation for such a distributed and heterogeneous environments. The need for robust identity management is paramount and, in this regard, Decentralised Identity Management (DIM) emerges as a promising solution. It leverages decentralised technologies to secure and facilitate identity interactions across the Computing Continuum. Particularly, to enhance security and privacy, it would be desirable to apply the principles of Self-Sovereign Identity (SSI). In this paradigm, users have full ownership and control of their digital identities that empowers individuals to manage and share their identity data on a need-to-know basis. These mechanisms could contribute to improve security properties during continuum resource management operations. In this context, this paper presents the design, workflows and implementation of a solution that provides authentication/authorisation features to distributed zero-trust based infrastructures across the continuum, enhancing security in resource sharing and resource acquisition stages. To this aim, the solution relies on key aspects like decentralisation, interoperability, trust management and privacy-enhancing capabilities. The decentralisation leverages distributed ledger technologies, such as blockchain, to establish a decentralised identity ecosystem. The solution prioritises interoperability, enabling nodes to seamlessly access and share their identities across different domains and environments. Trustworthiness is at the core of DIM, and privacy is also considered, incorporating privacy-preserving techniques that individuals to selectively disclose identity attributes while safeguarding sensitive information. The implementation includes different operations for allowing continuum frameworks to be enhanced with decentralised authentication and authorisation features. The performance has been evaluated measuring the impact for the adoption of the solution. The most expensive task, the self-identity generation, takes only a few seconds (in our deployment) and it is only executed once. Authorisation tasks operate in the millisecond range, which is a totally invaluable time if incorporated into resource acquisition processes in frameworks such as Liqo, used in the scope of FLUIDOS project.</p></div>","PeriodicalId":55132,"journal":{"name":"Future Generation Computer Systems-The International Journal of Escience","volume":null,"pages":null},"PeriodicalIF":6.2,"publicationDate":"2024-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0167739X24004291/pdfft?md5=b118fab0128173d8752d4ab90e0703c8&pid=1-s2.0-S0167739X24004291-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141915032","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
15+ years of joint parallel application performance analysis/tools training with Scalasca/Score-P and Paraver/Extrae toolsets 使用 Scalasca/Score-P 和 Paraver/Extrae 工具集进行 15 年以上的联合并行应用性能分析/工具培训
IF 6.2 2区 计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS Pub Date : 2024-08-02 DOI: 10.1016/j.future.2024.07.050

The diverse landscape of distributed heterogeneous computer systems currently available and being created to address computational challenges with the highest performance requirements presents daunting complexity for application developers. They must effectively decompose and distribute their application functionality and data, efficiently orchestrating the associated communication and synchronisation, on multi/manycore CPU processors with multiple attached acceleration devices structured within compute nodes with interconnection networks of various topologies.

Sophisticated compilers, runtime systems and libraries are (loosely) matched with debugging, performance measurement and analysis tools, with proprietary versions by integrators/vendors provided exclusively for their systems complemented by portable (primarily) open-source equivalents developed and supported by the international research community over many years. The Scalasca and Paraver toolsets are two widely employed examples of the latter, installed on personal notebook computers through to the largest leadership HPC systems. Over more than fifteen years their developers have worked closely together in numerous collaborative projects culminating in the creation of a universal parallel performance assessment and optimisation methodology focused on application execution efficiency and scalability, and the associated training and coaching of application developers (often in teams) in its productive use, reviewed in this article with lessons learnt therefrom.

为应对最高性能要求的计算挑战,目前可用和正在创建的分布式异构计算机系统种类繁多,这给应用开发人员带来了令人生畏的复杂性。他们必须在多核 CPU 处理器上有效地分解和分配其应用功能和数据,并有效地协调相关的通信和同步,同时在具有不同拓扑结构互连网络的计算节点内结构多个附加加速设备。
{"title":"15+ years of joint parallel application performance analysis/tools training with Scalasca/Score-P and Paraver/Extrae toolsets","authors":"","doi":"10.1016/j.future.2024.07.050","DOIUrl":"10.1016/j.future.2024.07.050","url":null,"abstract":"<div><p>The diverse landscape of distributed heterogeneous computer systems currently available and being created to address computational challenges with the highest performance requirements presents daunting complexity for application developers. They must effectively decompose and distribute their application functionality and data, efficiently orchestrating the associated communication and synchronisation, on multi/manycore CPU processors with multiple attached acceleration devices structured within compute nodes with interconnection networks of various topologies.</p><p>Sophisticated compilers, runtime systems and libraries are (loosely) matched with debugging, performance measurement and analysis tools, with proprietary versions by integrators/vendors provided exclusively for their systems complemented by portable (primarily) open-source equivalents developed and supported by the international research community over many years. The <em>Scalasca</em> and <em>Paraver</em> toolsets are two widely employed examples of the latter, installed on personal notebook computers through to the largest leadership HPC systems. Over more than fifteen years their developers have worked closely together in numerous collaborative projects culminating in the creation of a universal parallel performance assessment and optimisation methodology focused on application execution efficiency and scalability, and the associated training and coaching of application developers (often in teams) in its productive use, reviewed in this article with lessons learnt therefrom.</p></div>","PeriodicalId":55132,"journal":{"name":"Future Generation Computer Systems-The International Journal of Escience","volume":null,"pages":null},"PeriodicalIF":6.2,"publicationDate":"2024-08-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0167739X24004187/pdfft?md5=6d95fa0157a348afe6a2f74eeb9b1f7c&pid=1-s2.0-S0167739X24004187-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141915232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A cross-modal high-resolution image generation approach based on cloud-terminal collaboration for low-altitude intelligent network 基于云端协作的低空智能网络跨模态高分辨率图像生成方法
IF 6.2 2区 计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS Pub Date : 2024-08-02 DOI: 10.1016/j.future.2024.07.054

The advancement of digitization and automation in Low Altitude Intelligent Networking (LAIN) is constrained by limited computational resources and the absence of a dedicated modal transformation mechanism, affecting the performance of latency-sensitive missions. This study addresses these challenges by proposing a Downscaling Reconstruction Multi-scale Locally Focused Generative Adversarial Network (DR-MFGAN) with Federated Learning (FL). This integration employs wavelet transform downscaling and zero-shot residual learning techniques to create noise-suppressed image pairs, ultimately facilitating high-quality image reconstruction. The core network structure is composed of multidimensional residual blocks and generative confrontation network, and feature extraction is further enhanced through cross channel attention mechanism. Finally, distributed training based on Federated Learning ensures the training effectiveness of nodes with small data volumes.Experimental results demonstrate significant improvements: an 18.18% reduction in Mean Squared Error (MSE), a 33.52% increase in Peak Signal to Noise Ratio (PSNR), and a 39.54% improvement in Learned Perceptual Image Patch Similarity (LPIPS). The edge terminal can provide high-resolution imagery with limited data, achieving precise cross-modal transformations. This approach enhances LAIN capabilities, addressing computational and transformation challenges to support critical latency-sensitive missions.

低空智能网络(LAIN)数字化和自动化的发展受到有限计算资源和专用模态转换机制缺失的制约,影响了对延迟敏感的任务的性能。为应对这些挑战,本研究提出了具有联合学习(FL)功能的降尺度重构多尺度局部聚焦生成对抗网络(DR-MFGAN)。这种集成采用了小波变换降尺度和零镜头残差学习技术来创建噪声抑制图像对,最终促进高质量图像重建。核心网络结构由多维残差块和生成式对抗网络组成,并通过跨通道注意机制进一步加强特征提取。实验结果表明,这种方法有显著的改进:平均平方误差(MSE)降低了 18.18%,峰值信噪比(PSNR)提高了 33.52%,学习感知图像补丁相似度(LPIPS)提高了 39.54%。边缘终端可以用有限的数据提供高分辨率图像,实现精确的跨模态转换。这种方法增强了 LAIN 的能力,解决了计算和转换方面的难题,从而支持对延迟敏感的关键任务。
{"title":"A cross-modal high-resolution image generation approach based on cloud-terminal collaboration for low-altitude intelligent network","authors":"","doi":"10.1016/j.future.2024.07.054","DOIUrl":"10.1016/j.future.2024.07.054","url":null,"abstract":"<div><p>The advancement of digitization and automation in Low Altitude Intelligent Networking (LAIN) is constrained by limited computational resources and the absence of a dedicated modal transformation mechanism, affecting the performance of latency-sensitive missions. This study addresses these challenges by proposing a Downscaling Reconstruction Multi-scale Locally Focused Generative Adversarial Network (DR-MFGAN) with Federated Learning (FL). This integration employs wavelet transform downscaling and zero-shot residual learning techniques to create noise-suppressed image pairs, ultimately facilitating high-quality image reconstruction. The core network structure is composed of multidimensional residual blocks and generative confrontation network, and feature extraction is further enhanced through cross channel attention mechanism. Finally, distributed training based on Federated Learning ensures the training effectiveness of nodes with small data volumes.Experimental results demonstrate significant improvements: an 18.18% reduction in Mean Squared Error (MSE), a 33.52% increase in Peak Signal to Noise Ratio (PSNR), and a 39.54% improvement in Learned Perceptual Image Patch Similarity (LPIPS). The edge terminal can provide high-resolution imagery with limited data, achieving precise cross-modal transformations. This approach enhances LAIN capabilities, addressing computational and transformation challenges to support critical latency-sensitive missions.</p></div>","PeriodicalId":55132,"journal":{"name":"Future Generation Computer Systems-The International Journal of Escience","volume":null,"pages":null},"PeriodicalIF":6.2,"publicationDate":"2024-08-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141915070","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Self-adaptive asynchronous federated optimizer with adversarial sharpness-aware minimization 具有对抗性锐度感知最小化功能的自适应异步联合优化器
IF 6.2 2区 计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS Pub Date : 2024-07-31 DOI: 10.1016/j.future.2024.07.045

The past years have witnessed the success of a distributed learning system called Federated Learning (FL). Recently, asynchronous FL (AFL) has demonstrated its potential in concurrency compared to mainstream synchronous FL. However, the inherent systematic and statistical heterogeneity has presented several impediments to AFL: On the client side, the discrepancies in trips and local model drift impede global performance enhancement; On the server side, dynamic communication leads to significant fluctuations in gradient arrival time, while asynchronous arrival gradients with ambiguous value are not fully leveraged. In this paper, we propose an adaptive AFL framework, ARDAGH, which systematically addresses the aforementioned challenges: Firstly, to address the discrepancies in client trips, ARDAGH ensures their convergence by incorporating only 1-bit feedback information into the downlink. Secondly, to counter the drift of clients, ARDAGH generalizes the local models by employing our novel adversarial sharpness-aware minimization, which does not necessitate reliance on additional global variables. Thirdly, in the face of gradient latency issues, ARDAGH employs a communication-aware dropout strategy to adaptively compress gradients to ensure similar transmission times. Finally, to fully unleash the potential of each gradient, we establish a consistent optimal direction by conceptualizing the aggregation as an optimizer with successive momentum. In light of the comprehensive solution provided by ARDAGH, an algorithm named FedAMO is derived, and its superiority is confirmed by experimental results obtained under challenging prototype and simulation settings. Particularly in typical sentiment analysis tasks, FedAMO demonstrates an improvement of up to 5.351% with a 20.056-fold acceleration compared to conventional asynchronous methods.

过去几年中,一种名为 "联合学习"(FL)的分布式学习系统取得了成功。最近,与主流的同步学习系统相比,异步学习系统(AFL)在并发性方面显示出了潜力。然而,固有的系统和统计异质性给 AFL 带来了一些障碍:在客户端,行程差异和局部模型漂移阻碍了全局性能的提升;在服务器端,动态通信导致梯度到达时间大幅波动,而具有模糊值的异步到达梯度则无法充分利用。本文提出了一种自适应 AFL 框架 ARDAGH,系统地解决了上述难题:首先,针对客户端行程的差异,ARDAGH 通过在下行链路中仅加入 1 位反馈信息来确保其收敛。其次,为了应对客户端的漂移,ARDAGH 通过采用我们新颖的对抗性锐度感知最小化技术,对局部模型进行了扩展,从而无需依赖额外的全局变量。第三,面对梯度延迟问题,ARDAGH 采用了通信感知放弃策略,自适应地压缩梯度,以确保相似的传输时间。最后,为了充分发挥每个梯度的潜力,我们将聚合概念化为具有连续动力的优化器,从而建立了一致的优化方向。根据 ARDAGH 提供的综合解决方案,我们推导出了一种名为 FedAMO 的算法,并通过在具有挑战性的原型和模拟设置下获得的实验结果证实了该算法的优越性。特别是在典型的情感分析任务中,与传统的异步方法相比,FedAMO 的性能提高了 5.351%,加速了 20.056 倍。
{"title":"Self-adaptive asynchronous federated optimizer with adversarial sharpness-aware minimization","authors":"","doi":"10.1016/j.future.2024.07.045","DOIUrl":"10.1016/j.future.2024.07.045","url":null,"abstract":"<div><p>The past years have witnessed the success of a distributed learning system called Federated Learning (FL). Recently, asynchronous FL (AFL) has demonstrated its potential in concurrency compared to mainstream synchronous FL. However, the inherent systematic and statistical heterogeneity has presented several impediments to AFL: On the client side, the discrepancies in trips and local model drift impede global performance enhancement; On the server side, dynamic communication leads to significant fluctuations in gradient arrival time, while asynchronous arrival gradients with ambiguous value are not fully leveraged. In this paper, we propose an adaptive AFL framework, ARDAGH, which systematically addresses the aforementioned challenges: Firstly, to address the discrepancies in client trips, ARDAGH ensures their convergence by incorporating only 1-bit feedback information into the downlink. Secondly, to counter the drift of clients, ARDAGH generalizes the local models by employing our novel adversarial sharpness-aware minimization, which does not necessitate reliance on additional global variables. Thirdly, in the face of gradient latency issues, ARDAGH employs a communication-aware dropout strategy to adaptively compress gradients to ensure similar transmission times. Finally, to fully unleash the potential of each gradient, we establish a consistent optimal direction by conceptualizing the aggregation as an optimizer with successive momentum. In light of the comprehensive solution provided by ARDAGH, an algorithm named FedAMO is derived, and its superiority is confirmed by experimental results obtained under challenging prototype and simulation settings. Particularly in typical sentiment analysis tasks, FedAMO demonstrates an improvement of up to 5.351% with a 20.056-fold acceleration compared to conventional asynchronous methods.</p></div>","PeriodicalId":55132,"journal":{"name":"Future Generation Computer Systems-The International Journal of Escience","volume":null,"pages":null},"PeriodicalIF":6.2,"publicationDate":"2024-07-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141915134","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A study on characterizing energy, latency and security for Intrusion Detection Systems on heterogeneous embedded platforms 异构嵌入式平台上入侵检测系统的能量、延迟和安全性特征研究
IF 6.2 2区 计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS Pub Date : 2024-07-31 DOI: 10.1016/j.future.2024.07.051

Drone swarms are increasingly being used for critical missions and need to be protected against malicious users. Intrusion Detection Systems (IDS) are used to analyze network traffic in order to detect possible threats. Modern IDSs rely on machine learning models for this purpose. Optimizing the execution of resource-hungry IDS algorithms on resource-constrained drone devices, in terms of energy consumption, response time, memory footprint and guaranteed level of security, allows to extend the duration of missions. In addition, the embedded platforms used in drones often incorporate heterogeneous computing platforms on which IDSs could be executed. In this paper, we present a methodology and results about characterizing the execution of different IDS models on various processing elements, namely, Central Processing Units (CPU), Graphical Processing Units (GPU), Deep Learning Accelerators (DLA) and Field-Programmable Gate Array (FPGA). In effect, drones operate in different mission contexts in terms of criticality level, energy and memory budgets, and traffic load, so it is important to identify which IDS model to run on which processing element in a given context. For this sake, we evaluated several metrics on different platforms: energy and resource consumption, accuracy for malicious traffic detection and response time. Different models, namely Random Forests (RF), Convolutional Neural Networks (CNN) and Dense Neural Networks (DNN), have been implemented and characterized on different processing elements/platforms. This study has shown that relating the chosen implementation to the resources available on the drone is a judicious strategy to work on. It highlights the disparity between IDS implementations characteristics. For example, the inference time ranges from 1.27μs to 30 ms, the energy consumption per inference is between 10.7μJ and 70 mJ, and the accuracy of the IDS models is between 65.73% and 81.59%. In addition, we develop a set of guidelines for choosing the best IDS model given a mission context.

无人机群越来越多地用于执行关键任务,因此需要防范恶意用户。入侵检测系统(IDS)用于分析网络流量,以检测可能存在的威胁。现代 IDS 依靠机器学习模型来实现这一目的。在能源消耗、响应时间、内存占用和保证的安全级别方面,在资源受限的无人机设备上优化执行资源消耗大的 IDS 算法,可以延长任务的持续时间。此外,无人机中使用的嵌入式平台通常包含异构计算平台,可以在这些平台上执行 IDS。在本文中,我们介绍了在各种处理元件(即中央处理器(CPU)、图形处理单元(GPU)、深度学习加速器(DLA)和现场可编程门阵列(FPGA))上执行不同 IDS 模型的方法和结果。实际上,无人机在不同的任务环境中工作,其关键程度、能源和内存预算以及流量负载都不尽相同,因此,确定在特定环境中运行哪种 IDS 模型、在哪种处理元件上运行非常重要。为此,我们在不同平台上评估了几个指标:能源和资源消耗、恶意流量检测的准确性和响应时间。我们在不同的处理元件/平台上实施了不同的模型,即随机森林(RF)、卷积神经网络(CNN)和密集神经网络(DNN)。这项研究表明,将所选的实现方式与无人机上的可用资源联系起来是一种明智的工作策略。它突出了 IDS 实现特性之间的差异。例如,推理时间从 1.27μs 到 30 ms 不等,每次推理的能耗在 10.7μJ 到 70 mJ 之间,IDS 模型的准确率在 65.73% 到 81.59% 之间。此外,我们还开发了一套指南,用于在任务环境下选择最佳 IDS 模型。
{"title":"A study on characterizing energy, latency and security for Intrusion Detection Systems on heterogeneous embedded platforms","authors":"","doi":"10.1016/j.future.2024.07.051","DOIUrl":"10.1016/j.future.2024.07.051","url":null,"abstract":"<div><p>Drone swarms are increasingly being used for critical missions and need to be protected against malicious users. Intrusion Detection Systems (IDS) are used to analyze network traffic in order to detect possible threats. Modern IDSs rely on machine learning models for this purpose. Optimizing the execution of resource-hungry IDS algorithms on resource-constrained drone devices, in terms of energy consumption, response time, memory footprint and guaranteed level of security, allows to extend the duration of missions. In addition, the embedded platforms used in drones often incorporate heterogeneous computing platforms on which IDSs could be executed. In this paper, we present a methodology and results about characterizing the execution of different IDS models on various processing elements, namely, Central Processing Units (CPU), Graphical Processing Units (GPU), Deep Learning Accelerators (DLA) and Field-Programmable Gate Array (FPGA). In effect, drones operate in different mission contexts in terms of criticality level, energy and memory budgets, and traffic load, so it is important to identify which IDS model to run on which processing element in a given context. For this sake, we evaluated several metrics on different platforms: energy and resource consumption, accuracy for malicious traffic detection and response time. Different models, namely Random Forests (RF), Convolutional Neural Networks (CNN) and Dense Neural Networks (DNN), have been implemented and characterized on different processing elements/platforms. This study has shown that relating the chosen implementation to the resources available on the drone is a judicious strategy to work on. It highlights the disparity between IDS implementations characteristics. For example, the inference time ranges from <span><math><mrow><mn>1</mn><mo>.</mo><mn>27</mn><mspace></mspace><mi>μ</mi><mi>s</mi></mrow></math></span> to 30 ms, the energy consumption per inference is between <span><math><mrow><mn>10</mn><mo>.</mo><mn>7</mn><mspace></mspace><mi>μ</mi><mi>J</mi></mrow></math></span> and 70 mJ, and the accuracy of the IDS models is between 65.73% and 81.59%. In addition, we develop a set of guidelines for choosing the best IDS model given a mission context.</p></div>","PeriodicalId":55132,"journal":{"name":"Future Generation Computer Systems-The International Journal of Escience","volume":null,"pages":null},"PeriodicalIF":6.2,"publicationDate":"2024-07-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141979544","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Harnessing federated learning for anomaly detection in supercomputer nodes 利用联合学习进行超级计算机节点异常检测
IF 6.2 2区 计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS Pub Date : 2024-07-31 DOI: 10.1016/j.future.2024.07.052

High-performance computing (HPC) systems are a crucial component of modern society, with a significant impact in areas ranging from economics to scientific research, thanks to their unrivaled computational capabilities. For this reason, the worldwide HPC installation is steeply trending upwards, with no sign of slowing down. However, these machines are both complex, comprising millions of heterogeneous components, hard to effectively manage, and very costly (both in terms of economic investment and of energy consumption). Therefore, maximizing their productivity is of paramount importance. For instance, anomalies and faults can generate significant downtime due to the difficulty of promptly detecting them, as there are potentially many sources of issues preventing the correct functioning of computing nodes.

In recent years, several data-driven methods have been proposed to automatically detect anomalies in HPC systems, exploiting the fact that modern supercomputers are typically endowed with fine-grained monitoring infrastructures, collecting data that can be used to characterize the system behavior. Thus, it is possible to teach Machine Learning (ML) models to distinguish normal and anomalous states automatically. In this paper, we contribute to this line of research with a novel intuition, namely exploiting Federated Learning (FL) to improve the accuracy of anomaly detection models for HPC nodes. Although FL is not typically exploited in the HPC context, we show that FL can boost several types of underlying ML models, from supervised to unsupervised ones. We demonstrate our approach on a production Tier-0 supercomputer hosted in Italy. Applying FL to anomaly detection improves the average f-score from 0.46 to 0.87. Our research also shows FL can reduce the data collection time required to develop a representation data set, facilitating faster deployment of anomaly detection models. ML models need 5 months of training data for efficient anomaly detection performance while using FL reduces the training set by 15 times to 1.25 weeks.

高性能计算(HPC)系统是现代社会的重要组成部分,凭借其无与伦比的计算能力,在从经济到科学研究等各个领域都产生了重大影响。因此,全球高性能计算系统的安装量呈急剧上升趋势,而且没有任何放缓的迹象。然而,这些机器非常复杂,由数百万个异构组件组成,难以有效管理,而且成本非常高昂(包括经济投资和能源消耗)。因此,最大限度地提高它们的生产率至关重要。例如,由于难以及时发现异常和故障,可能会导致大量停机时间,因为阻碍计算节点正常运行的潜在问题来源很多。
{"title":"Harnessing federated learning for anomaly detection in supercomputer nodes","authors":"","doi":"10.1016/j.future.2024.07.052","DOIUrl":"10.1016/j.future.2024.07.052","url":null,"abstract":"<div><p>High-performance computing (HPC) systems are a crucial component of modern society, with a significant impact in areas ranging from economics to scientific research, thanks to their unrivaled computational capabilities. For this reason, the worldwide HPC installation is steeply trending upwards, with no sign of slowing down. However, these machines are both complex, comprising millions of heterogeneous components, hard to effectively manage, and very costly (both in terms of economic investment and of energy consumption). Therefore, maximizing their productivity is of paramount importance. For instance, anomalies and faults can generate significant downtime due to the difficulty of promptly detecting them, as there are potentially many sources of issues preventing the correct functioning of computing nodes.</p><p>In recent years, several data-driven methods have been proposed to automatically detect anomalies in HPC systems, exploiting the fact that modern supercomputers are typically endowed with fine-grained monitoring infrastructures, collecting data that can be used to characterize the system behavior. Thus, it is possible to teach Machine Learning (ML) models to distinguish normal and anomalous states automatically. In this paper, we contribute to this line of research with a novel intuition, namely exploiting Federated Learning (FL) to improve the accuracy of anomaly detection models for HPC nodes. Although FL is not typically exploited in the HPC context, we show that FL can boost several types of underlying ML models, from supervised to unsupervised ones. We demonstrate our approach on a production Tier-0 supercomputer hosted in Italy. Applying FL to anomaly detection improves the average f-score from 0.46 to 0.87. Our research also shows FL can reduce the data collection time required to develop a representation data set, facilitating faster deployment of anomaly detection models. ML models need 5 months of training data for efficient anomaly detection performance while using FL reduces the training set by 15 times to 1.25 weeks.</p></div>","PeriodicalId":55132,"journal":{"name":"Future Generation Computer Systems-The International Journal of Escience","volume":null,"pages":null},"PeriodicalIF":6.2,"publicationDate":"2024-07-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141915220","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Future Generation Computer Systems-The International Journal of Escience
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1