Pub Date : 2020-11-01DOI: 10.1109/ICDCS47774.2020.00148
Xinyu Lei, Tian Xie, Guan-Hua Tu, A. Liu
In recent years, the Bitcoin (BTC) payment is increasingly popular in retailers and service providers. A BTC transaction (tx) needs six confirmations (one hour) to be validated, making it not suitable for fast-pay scenarios. Theoretically, a shorter waiting time period increases the success possibility of a double-spending attack. To address this problem, we propose BTCFast scheme to support fast BTC tx. BTCFast is a novel, decentralized, escrow-based scheme on top of the programmable smart contract (PSC)-enabled blockchains (e.g. Ethereum, EOS). We develop a smart contract (PayJudger) to work as a trusted payment judger, which guarantees the tx fairness. In addition, we devise a proof-of-work (PoW)-based payment judgment mechanism for PayJudger to resolve a BTC payment dispute. Our theoretical and experimental results show that BTCFast can reduce the waiting time to be less than 1 second with comparable security as the current approach (i.e., waiting for six confirmations) with no extra operation fee.
{"title":"An Inter-blockchain Escrow Approach for Fast Bitcoin Payment","authors":"Xinyu Lei, Tian Xie, Guan-Hua Tu, A. Liu","doi":"10.1109/ICDCS47774.2020.00148","DOIUrl":"https://doi.org/10.1109/ICDCS47774.2020.00148","url":null,"abstract":"In recent years, the Bitcoin (BTC) payment is increasingly popular in retailers and service providers. A BTC transaction (tx) needs six confirmations (one hour) to be validated, making it not suitable for fast-pay scenarios. Theoretically, a shorter waiting time period increases the success possibility of a double-spending attack. To address this problem, we propose BTCFast scheme to support fast BTC tx. BTCFast is a novel, decentralized, escrow-based scheme on top of the programmable smart contract (PSC)-enabled blockchains (e.g. Ethereum, EOS). We develop a smart contract (PayJudger) to work as a trusted payment judger, which guarantees the tx fairness. In addition, we devise a proof-of-work (PoW)-based payment judgment mechanism for PayJudger to resolve a BTC payment dispute. Our theoretical and experimental results show that BTCFast can reduce the waiting time to be less than 1 second with comparable security as the current approach (i.e., waiting for six confirmations) with no extra operation fee.","PeriodicalId":158630,"journal":{"name":"2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130948832","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-11-01DOI: 10.1109/ICDCS47774.2020.00061
T. Pulkkinen, J. Nurminen, P. Nurmi
WiFi networks are increasingly subjected to cross-technology interference with emerging IoT and even mobile communication solutions all crowding the 2.4 GHz ISM band where WiFi networks conventionally operate. Due to the diversity of interference sources, maintaining high level of network performance is becoming increasing difficult. Recently, deep learning based interference detection has been proposed as a potentially powerful way to identify sources of interference and to provide feedback on how to mitigate their effects. The performance of such approaches has been shown to be impressive in controlled evaluations. However, little information exists on how they generalize to the complexity of everyday environments. In this paper, we contribute by conducting a comprehensive performance evaluation of deep learning based interference detection. In our evaluation, we consider five orthogonal but complementary metrics: correctness, overfitting, robustness, efficiency, and interpretability. Our results show that, while deep learning indeed has excellent correctness (i.e., detection accuracy), it can be prone to noise in measurements (e.g., struggle when transmission power is dynamically adjusted) and suffers from poor interpretability. Deep learning is also highly sensitive to the quality and quantity of training data, with performance decreasing rapidly when the training and testing measurements come from environments with different characteristics. To compensate for weaknesses of deep learning, as our second contribution we propose a novel signal modeling approach for interference detection and compare it against deep learning. Our results demonstrate that, in terms of errors, there are some differences across the two approaches, with signal modeling being better at identifying technologies that rely on frequency hopping or that have dynamic spectrum signatures but suffering in other cases. Based on our results, we draw guidelines for improving interference detection performance.
{"title":"Understanding WiFi Cross-Technology Interference Detection in the Real World","authors":"T. Pulkkinen, J. Nurminen, P. Nurmi","doi":"10.1109/ICDCS47774.2020.00061","DOIUrl":"https://doi.org/10.1109/ICDCS47774.2020.00061","url":null,"abstract":"WiFi networks are increasingly subjected to cross-technology interference with emerging IoT and even mobile communication solutions all crowding the 2.4 GHz ISM band where WiFi networks conventionally operate. Due to the diversity of interference sources, maintaining high level of network performance is becoming increasing difficult. Recently, deep learning based interference detection has been proposed as a potentially powerful way to identify sources of interference and to provide feedback on how to mitigate their effects. The performance of such approaches has been shown to be impressive in controlled evaluations. However, little information exists on how they generalize to the complexity of everyday environments. In this paper, we contribute by conducting a comprehensive performance evaluation of deep learning based interference detection. In our evaluation, we consider five orthogonal but complementary metrics: correctness, overfitting, robustness, efficiency, and interpretability. Our results show that, while deep learning indeed has excellent correctness (i.e., detection accuracy), it can be prone to noise in measurements (e.g., struggle when transmission power is dynamically adjusted) and suffers from poor interpretability. Deep learning is also highly sensitive to the quality and quantity of training data, with performance decreasing rapidly when the training and testing measurements come from environments with different characteristics. To compensate for weaknesses of deep learning, as our second contribution we propose a novel signal modeling approach for interference detection and compare it against deep learning. Our results demonstrate that, in terms of errors, there are some differences across the two approaches, with signal modeling being better at identifying technologies that rely on frequency hopping or that have dynamic spectrum signatures but suffering in other cases. Based on our results, we draw guidelines for improving interference detection performance.","PeriodicalId":158630,"journal":{"name":"2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS)","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121802388","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Because automatic recovery from failures is of great importance for future operations of ICT systems, we propose a framework for learning a recovery policy using deep reinforcement learning. In our framework, while iteratively trying various recovery actions and observing system metrics in a target system, an agent autonomously learns the optimal recovery policy, which indicates what recovery action should be executed on the basis of observations. By using failure injection tools designed for Chaos Engineering, we can reproduce many types of failures in the target system, thereby making the agent learn a recovery policy applicable to various failures. Once the recovery policy is obtained, we can automate failure recovery by executing recovery actions that the recovery policy returns. Unlike most previous methods, our framework does not require any historical documents of failure recovery or modeling of system behavior. To verify the feasibility of the framework, we conducted an experiment using a container-based environment built on a Kubernetes cluster, demonstrating that training converges in a few days and the obtained recovery policy can successfully recover from failures with a minimum number of recovery actions.
{"title":"A Framework for Automatic Failure Recovery in ICT Systems by Deep Reinforcement Learning","authors":"Hiroki Ikeuchi, Jiawen Ge, Yoichi Matsuo, Keishiro Watanabe","doi":"10.1109/ICDCS47774.2020.00170","DOIUrl":"https://doi.org/10.1109/ICDCS47774.2020.00170","url":null,"abstract":"Because automatic recovery from failures is of great importance for future operations of ICT systems, we propose a framework for learning a recovery policy using deep reinforcement learning. In our framework, while iteratively trying various recovery actions and observing system metrics in a target system, an agent autonomously learns the optimal recovery policy, which indicates what recovery action should be executed on the basis of observations. By using failure injection tools designed for Chaos Engineering, we can reproduce many types of failures in the target system, thereby making the agent learn a recovery policy applicable to various failures. Once the recovery policy is obtained, we can automate failure recovery by executing recovery actions that the recovery policy returns. Unlike most previous methods, our framework does not require any historical documents of failure recovery or modeling of system behavior. To verify the feasibility of the framework, we conducted an experiment using a container-based environment built on a Kubernetes cluster, demonstrating that training converges in a few days and the obtained recovery policy can successfully recover from failures with a minimum number of recovery actions.","PeriodicalId":158630,"journal":{"name":"2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS)","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133999997","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-11-01DOI: 10.1109/ICDCS47774.2020.00167
Minghui Dai, Zhou Su, Jiliang Li, Jian Zhou
As the ever-increasing capacities of internet of things (IoT), unmanned aerial vehicle (UAV)-assisted IoT becomes a promising paradigm for improving network connectivity, extending the coverage of network and computing offloading. However, due to the limitation of battery lifetime and computing capacities of UAVs, the offloading scheme for UAVs presents a new challenge in IoT. Therefore, in this paper, an energy-efficient edge offloading scheme is proposed to improve the offloading efficiency of UAVs. Firstly, based on the data transmission delay of UAVs and computing delay of edge nodes, the matching scheme is designed to obtain the optimal matching between UAVs and edge nodes. Secondly, the energy-efficient offloading scheme for UAVs and edge nodes is modeled as a bargaining game. Then, the offloading strategy based on incentive algorithm is developed to improve the offloading efficiency. Finally, the simulation results demonstrate that the proposed offloading scheme can significantly promote the effectiveness of offloading compared with the conventional schemes.
{"title":"An Energy-Efficient Edge Offloading Scheme for UAV-Assisted Internet of Things","authors":"Minghui Dai, Zhou Su, Jiliang Li, Jian Zhou","doi":"10.1109/ICDCS47774.2020.00167","DOIUrl":"https://doi.org/10.1109/ICDCS47774.2020.00167","url":null,"abstract":"As the ever-increasing capacities of internet of things (IoT), unmanned aerial vehicle (UAV)-assisted IoT becomes a promising paradigm for improving network connectivity, extending the coverage of network and computing offloading. However, due to the limitation of battery lifetime and computing capacities of UAVs, the offloading scheme for UAVs presents a new challenge in IoT. Therefore, in this paper, an energy-efficient edge offloading scheme is proposed to improve the offloading efficiency of UAVs. Firstly, based on the data transmission delay of UAVs and computing delay of edge nodes, the matching scheme is designed to obtain the optimal matching between UAVs and edge nodes. Secondly, the energy-efficient offloading scheme for UAVs and edge nodes is modeled as a bargaining game. Then, the offloading strategy based on incentive algorithm is developed to improve the offloading efficiency. Finally, the simulation results demonstrate that the proposed offloading scheme can significantly promote the effectiveness of offloading compared with the conventional schemes.","PeriodicalId":158630,"journal":{"name":"2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS)","volume":"7 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"113932147","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-11-01DOI: 10.1109/ICDCS47774.2020.00195
J. Gunasekaran, P. Thinakaran, N. Nachiappan, R. Kannan, M. Kandemir, C. Das
Datacenters are witnessing an increasing trend in adopting microservice-based architecture for application design, which consists of a combination of different microservices. Typically these applications are short-lived and are administered with strict Service Level Objective (SLO) requirements. Traditional virtual machine (VM) based provisioning for such applications not only suffers from long latency when provisioning resources (as VMs tend to take a few minutes to start up), but also places an additional overhead of server management and provisioning on the users. This led to the adoption of serverless functions, where applications are composed as functions and hosted in containers. However, state-of-the-art schedulers employed in serverless platforms tend to look at microservice-based applications similar to conventional monolithic black-box applications. To detect all the inefficiencies, we characterize the end-to-end life cycle of these microservice-based applications in this work. Our findings show that the applications suffer from poor scheduling of microservices due to reactive container provisioning during workload fluctuations, thereby resulting in either in SLO violations or colossal container over-provisioning, in turn leading to poor resource utilization. We also find that there is an ample amount of slack available at each stage of application execution, which can potentially be leveraged to improve the overall application performance.
{"title":"Characterizing Bottlenecks in Scheduling Microservices on Serverless Platforms","authors":"J. Gunasekaran, P. Thinakaran, N. Nachiappan, R. Kannan, M. Kandemir, C. Das","doi":"10.1109/ICDCS47774.2020.00195","DOIUrl":"https://doi.org/10.1109/ICDCS47774.2020.00195","url":null,"abstract":"Datacenters are witnessing an increasing trend in adopting microservice-based architecture for application design, which consists of a combination of different microservices. Typically these applications are short-lived and are administered with strict Service Level Objective (SLO) requirements. Traditional virtual machine (VM) based provisioning for such applications not only suffers from long latency when provisioning resources (as VMs tend to take a few minutes to start up), but also places an additional overhead of server management and provisioning on the users. This led to the adoption of serverless functions, where applications are composed as functions and hosted in containers. However, state-of-the-art schedulers employed in serverless platforms tend to look at microservice-based applications similar to conventional monolithic black-box applications. To detect all the inefficiencies, we characterize the end-to-end life cycle of these microservice-based applications in this work. Our findings show that the applications suffer from poor scheduling of microservices due to reactive container provisioning during workload fluctuations, thereby resulting in either in SLO violations or colossal container over-provisioning, in turn leading to poor resource utilization. We also find that there is an ample amount of slack available at each stage of application execution, which can potentially be leveraged to improve the overall application performance.","PeriodicalId":158630,"journal":{"name":"2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS)","volume":"175 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127673246","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-11-01DOI: 10.1109/ICDCS47774.2020.00104
Ellen Mitsopoulou, Juliana Litou, V. Kalogeraki
In this work we aim to provide an efficient solution to the problem of online task allocation in spatial crowdsourcing systems. We focus on the objectives of platform utility maximization and worker utility maximization, yet the proposed schema is generic enough to accommodate more objectives. The goal is to find an allocation of tasks to workers that maximizes the platform’s profit and reliability of the results, while simultaneously assigns tasks based on the users’ interests to increase user engagement and hence the probability that the users will complete the tasks on time. Our scheme works well in highly fluctuating environments where the tasks to be executed require that the workers meet certain criteria of expertise, availability, reliability, etc. Our detailed experimental evaluation illustrates the benefits and practicality of our approach and demonstrates that our approach outperforms its competitors.
{"title":"Multi-Objective Online Task Allocation in Spatial Crowdsourcing Systems","authors":"Ellen Mitsopoulou, Juliana Litou, V. Kalogeraki","doi":"10.1109/ICDCS47774.2020.00104","DOIUrl":"https://doi.org/10.1109/ICDCS47774.2020.00104","url":null,"abstract":"In this work we aim to provide an efficient solution to the problem of online task allocation in spatial crowdsourcing systems. We focus on the objectives of platform utility maximization and worker utility maximization, yet the proposed schema is generic enough to accommodate more objectives. The goal is to find an allocation of tasks to workers that maximizes the platform’s profit and reliability of the results, while simultaneously assigns tasks based on the users’ interests to increase user engagement and hence the probability that the users will complete the tasks on time. Our scheme works well in highly fluctuating environments where the tasks to be executed require that the workers meet certain criteria of expertise, availability, reliability, etc. Our detailed experimental evaluation illustrates the benefits and practicality of our approach and demonstrates that our approach outperforms its competitors.","PeriodicalId":158630,"journal":{"name":"2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127675962","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
While deep neural networks (DNNs) have led to a paradigm shift, its exorbitant computational requirement has always been a roadblock in its deployment to the edge, such as wearable devices and smartphones. Hence a hybrid edge-cloud computational framework is proposed to transfer part of the computation to the cloud, by naively partitioning the DNN operations under the constant network condition assumption. However, real-world network state varies greatly depending on the context, and DNN partitioning only has limited strategy space. In this paper, we explore the structural flexibility of DNN to fit the edge model to varying network contexts and different deployment platforms. Specifically, we designed a reinforcement learning-based decision engine to search for model transformation strategies in response to a combined objective of model accuracy and computation latency. The engine generates a context-aware model tree so that the DNN can decide the model branch to switch to at runtime. By the emulation and field experimental results, our approach enjoys a 30% − 50% latency reduction while retaining the model accuracy.
{"title":"Context-Aware Deep Model Compression for Edge Cloud Computing","authors":"Lingdong Wang, Liyao Xiang, Jiayu Xu, Jiaju Chen, Xing Zhao, Dixi Yao, Xinbing Wang, Baochun Li","doi":"10.1109/ICDCS47774.2020.00101","DOIUrl":"https://doi.org/10.1109/ICDCS47774.2020.00101","url":null,"abstract":"While deep neural networks (DNNs) have led to a paradigm shift, its exorbitant computational requirement has always been a roadblock in its deployment to the edge, such as wearable devices and smartphones. Hence a hybrid edge-cloud computational framework is proposed to transfer part of the computation to the cloud, by naively partitioning the DNN operations under the constant network condition assumption. However, real-world network state varies greatly depending on the context, and DNN partitioning only has limited strategy space. In this paper, we explore the structural flexibility of DNN to fit the edge model to varying network contexts and different deployment platforms. Specifically, we designed a reinforcement learning-based decision engine to search for model transformation strategies in response to a combined objective of model accuracy and computation latency. The engine generates a context-aware model tree so that the DNN can decide the model branch to switch to at runtime. By the emulation and field experimental results, our approach enjoys a 30% − 50% latency reduction while retaining the model accuracy.","PeriodicalId":158630,"journal":{"name":"2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129084819","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-11-01DOI: 10.1109/ICDCS47774.2020.00019
Vipul Gupta, Dominic Carrano, Yaoqing Yang, Vaishaal Shankar, T. Courtade, K. Ramchandran
Inexpensive cloud services, such as serverless computing, are often vulnerable to straggling nodes that increase the end-to-end latency for distributed computation. We propose and implement simple yet principled approaches for straggler mitigation in serverless systems for matrix multiplication and evaluate them on several common applications from machine learning and high-performance computing. The proposed schemes are inspired by error-correcting codes and employ parallel encoding and decoding over the data stored in the cloud using serverless workers. This creates a fully distributed computing framework without using a master node to conduct encoding or decoding, which removes the computation, communication and storage bottleneck at the master. On the theory side, we establish that our proposed scheme is asymptotically optimal in terms of decoding time and provide a lower bound on the number of stragglers it can tolerate with high probability. Through extensive experiments, we show that our scheme outperforms existing schemes such as speculative execution and other coding theoretic methods by at least 25%.
{"title":"Serverless Straggler Mitigation using Error-Correcting Codes","authors":"Vipul Gupta, Dominic Carrano, Yaoqing Yang, Vaishaal Shankar, T. Courtade, K. Ramchandran","doi":"10.1109/ICDCS47774.2020.00019","DOIUrl":"https://doi.org/10.1109/ICDCS47774.2020.00019","url":null,"abstract":"Inexpensive cloud services, such as serverless computing, are often vulnerable to straggling nodes that increase the end-to-end latency for distributed computation. We propose and implement simple yet principled approaches for straggler mitigation in serverless systems for matrix multiplication and evaluate them on several common applications from machine learning and high-performance computing. The proposed schemes are inspired by error-correcting codes and employ parallel encoding and decoding over the data stored in the cloud using serverless workers. This creates a fully distributed computing framework without using a master node to conduct encoding or decoding, which removes the computation, communication and storage bottleneck at the master. On the theory side, we establish that our proposed scheme is asymptotically optimal in terms of decoding time and provide a lower bound on the number of stragglers it can tolerate with high probability. Through extensive experiments, we show that our scheme outperforms existing schemes such as speculative execution and other coding theoretic methods by at least 25%.","PeriodicalId":158630,"journal":{"name":"2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS)","volume":"1961 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129363302","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-11-01DOI: 10.1109/ICDCS47774.2020.00062
Hanlin Lu, T. He, Shiqiang Wang, Changchang Liu, M. Mahdavi, V. Narayanan, Kevin S. Chan, Stephen Pasteris
We consider the problem of computing the k-means centers for a large high-dimensional dataset in the context of edge-based machine learning, where data sources offload machine learning computation to nearby edge servers. k-Means computation is fundamental to many data analytics, and the capability of computing provably accurate k-means centers by leveraging the computation power of the edge servers, at a low communication and computation cost to the data sources, will greatly improve the performance of these analytics. We propose to let the data sources send small summaries, generated by joint dimensionality reduction (DR) and cardinality reduction (CR), to support approximate k-means computation at reduced complexity and communication cost. By analyzing the complexity, the communication cost, and the approximation error of k-means algorithms based on state-of-the-art DR/CR methods, we show that: (i) in the single-source case, it is possible to achieve a near-optimal approximation at a near-linear complexity and a constant communication cost, (ii) in the multiple-source case, it is possible to achieve similar performance at a logarithmic communication cost, and (iii) the order of applying DR and CR significantly affects the complexity and the communication cost. Our findings are validated through experiments based on real datasets.
{"title":"Communication-efficient k-Means for Edge-based Machine Learning","authors":"Hanlin Lu, T. He, Shiqiang Wang, Changchang Liu, M. Mahdavi, V. Narayanan, Kevin S. Chan, Stephen Pasteris","doi":"10.1109/ICDCS47774.2020.00062","DOIUrl":"https://doi.org/10.1109/ICDCS47774.2020.00062","url":null,"abstract":"We consider the problem of computing the k-means centers for a large high-dimensional dataset in the context of edge-based machine learning, where data sources offload machine learning computation to nearby edge servers. k-Means computation is fundamental to many data analytics, and the capability of computing provably accurate k-means centers by leveraging the computation power of the edge servers, at a low communication and computation cost to the data sources, will greatly improve the performance of these analytics. We propose to let the data sources send small summaries, generated by joint dimensionality reduction (DR) and cardinality reduction (CR), to support approximate k-means computation at reduced complexity and communication cost. By analyzing the complexity, the communication cost, and the approximation error of k-means algorithms based on state-of-the-art DR/CR methods, we show that: (i) in the single-source case, it is possible to achieve a near-optimal approximation at a near-linear complexity and a constant communication cost, (ii) in the multiple-source case, it is possible to achieve similar performance at a logarithmic communication cost, and (iii) the order of applying DR and CR significantly affects the complexity and the communication cost. Our findings are validated through experiments based on real datasets.","PeriodicalId":158630,"journal":{"name":"2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS)","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117350266","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2020-11-01DOI: 10.1109/ICDCS47774.2020.00173
Seunghyun Lee, Seokho Son, Jungsu Han, JongWon Kim
In the cloud field, there is an increasing demand for globalized services and corresponding execution environments that overcome local limitations and selectively utilize optimal resources. Utilizing multi-cloud deployments and operations rather than using a single cloud is an effective way to satisfy the increasing demand. In particular, we need to provide cloud-native environment to organically support services based on a microservices architecture. In this paper, we propose a cloud-native workload profiling system with Kubernetes-orchestrated multi-cluster configuration. The contributions of this paper are as follows. (i) We design the operating software over multiple cloud-native cluster to select optimal resources by monitoring. (ii) For operating the multiple clusters through the design, we define and design specific general service workloads. Also, we implement the workloads in application software (iii) To seek optimal resources, we deployed the general workloads and monitored resource usage repeatedly in detail. We calculate resource variation in comparison with initial resource usage and average resource usage after deploying the service workloads. Also, we analyze the resource monitoring result. We expect this methodology can find proper resources for service workload types.
{"title":"Refining Micro Services Placement over Multiple Kubernetes-orchestrated Clusters employing Resource Monitoring","authors":"Seunghyun Lee, Seokho Son, Jungsu Han, JongWon Kim","doi":"10.1109/ICDCS47774.2020.00173","DOIUrl":"https://doi.org/10.1109/ICDCS47774.2020.00173","url":null,"abstract":"In the cloud field, there is an increasing demand for globalized services and corresponding execution environments that overcome local limitations and selectively utilize optimal resources. Utilizing multi-cloud deployments and operations rather than using a single cloud is an effective way to satisfy the increasing demand. In particular, we need to provide cloud-native environment to organically support services based on a microservices architecture. In this paper, we propose a cloud-native workload profiling system with Kubernetes-orchestrated multi-cluster configuration. The contributions of this paper are as follows. (i) We design the operating software over multiple cloud-native cluster to select optimal resources by monitoring. (ii) For operating the multiple clusters through the design, we define and design specific general service workloads. Also, we implement the workloads in application software (iii) To seek optimal resources, we deployed the general workloads and monitored resource usage repeatedly in detail. We calculate resource variation in comparison with initial resource usage and average resource usage after deploying the service workloads. Also, we analyze the resource monitoring result. We expect this methodology can find proper resources for service workload types.","PeriodicalId":158630,"journal":{"name":"2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS)","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115773452","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：