Seep Goel, Abhishek Singh, R. Garg, Mudit Verma, P. Jayachandran
In this paper, we consider the problem of fair scheduling of transactions of multiple types that are submitted to a permissioned blockchain system. Permissioned blockchains are being increasingly used for enterprise applications and by design are heterogeneous in nature, with different peer organizations performing different business functions. Transactions execute different smart contract operations that may have widely varying business importance. In such a setting, we argue that the typically adopted First-In-First-Out ordering mechanism for transactions in a blockchain system, which is a performance-limited resource, is inefficient and unfair. We propose a weighted fair queueing strategy for ordering transactions that can support differentiated quality of service for submitted transactions on the blockchain. The main challenge we address in this paper is to support fair allocation and differentiation in a decentralized manner, as there is no single authority that can facilitate this as in traditional systems. We demonstrate such a fair scheduling strategy and support multiple transaction types with different priorities on Hyperledger Fabric.
{"title":"Resource Fairness and Prioritization of Transactions in Permissioned Blockchain Systems (Industry Track)","authors":"Seep Goel, Abhishek Singh, R. Garg, Mudit Verma, P. Jayachandran","doi":"10.1145/3284028.3284035","DOIUrl":"https://doi.org/10.1145/3284028.3284035","url":null,"abstract":"In this paper, we consider the problem of fair scheduling of transactions of multiple types that are submitted to a permissioned blockchain system. Permissioned blockchains are being increasingly used for enterprise applications and by design are heterogeneous in nature, with different peer organizations performing different business functions. Transactions execute different smart contract operations that may have widely varying business importance. In such a setting, we argue that the typically adopted First-In-First-Out ordering mechanism for transactions in a blockchain system, which is a performance-limited resource, is inefficient and unfair. We propose a weighted fair queueing strategy for ordering transactions that can support differentiated quality of service for submitted transactions on the blockchain. The main challenge we address in this paper is to support fair allocation and differentiation in a decentralized manner, as there is no single authority that can facilitate this as in traditional systems. We demonstrate such a fair scheduling strategy and support multiple transaction types with different priorities on Hyperledger Fabric.","PeriodicalId":285212,"journal":{"name":"Proceedings of the 19th International Middleware Conference Industry","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116003561","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Containers and microservices are dominating the world of data center and cloud computing. As the scale, dynamism and complexity grow, the performance of the DNS system in container clusters becomes vital. As the world's third and China's largest e-commerce site by revenue, JD.com runs one of the world's largest Kubernetes container clusters in production. It is imperative that the DNS system can handle extremely high traffic. In this paper, we present ContainerDNS, a high performance DNS system for very large scale container clusters with millions of containers. ContainerDNS maximizes DNS system performance and scalability by optimizing DNS packet processing and using efficient memory and cache management. ContainerDNS has been deployed in JD's container platform with 30,000 servers and 500,000 containers running tens of thousands of services and applications. It improves the maximum throughput from 130,000 to 9,000,000 QPS, a 67X performance boost comparing to existing DNS systems.
{"title":"A High Performance, Scalable DNS Service for Very Large Scale Container Cloud Platforms","authors":"Haifeng Liu, Shugang Chen, Yongcheng Bao, Wanli Yang, Yuan Chen, Wei Ding, Huasong Shan","doi":"10.1145/3284028.3284034","DOIUrl":"https://doi.org/10.1145/3284028.3284034","url":null,"abstract":"Containers and microservices are dominating the world of data center and cloud computing. As the scale, dynamism and complexity grow, the performance of the DNS system in container clusters becomes vital. As the world's third and China's largest e-commerce site by revenue, JD.com runs one of the world's largest Kubernetes container clusters in production. It is imperative that the DNS system can handle extremely high traffic. In this paper, we present ContainerDNS, a high performance DNS system for very large scale container clusters with millions of containers. ContainerDNS maximizes DNS system performance and scalability by optimizing DNS packet processing and using efficient memory and cache management. ContainerDNS has been deployed in JD's container platform with 30,000 servers and 500,000 containers running tens of thousands of services and applications. It improves the maximum throughput from 130,000 to 9,000,000 QPS, a 67X performance boost comparing to existing DNS systems.","PeriodicalId":285212,"journal":{"name":"Proceedings of the 19th International Middleware Conference Industry","volume":"2018 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134482059","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Rémi Canillas, Rania Talbi, S. Bouchenak, Omar Hasan, L. Brunie, Laurent Sarrat
With the wide adoption of the Internet, digital transactions surge exponentially and so do the impersonation fraud. While machine learning techniques show strong promise to be the building block for digital fraud detection systems, clients may be reluctant to share the raw data with such systems due to privacy concerns. The emerging privacy preserving machine learning techniques that employ homomorphic encryption to resolve this conundrum unfortunately increases the computational overhead of detection. In this paper, we present a first-of-a-kind empirical performance study of a private fraud detection system developed at SiS ID, a French business security platform. A privacy-preserving decision tree which can classify transactions into four risk classes (safe, moderately risky, very risky and fraud) is trained on more than 160000 real world transactions, and we quantitatively compare the classification accuracy, latency and network bandwidth under various combinations of encryption parameters and learning hyper-parameters, in order to explore the impact of the configuration on the performances. Our results show that the computation and communication overhead of processing encrypted data increases by an order of magnitude of 5, and highly depends on the configuration of the encryption key and the number of nodes in the decision tree.
{"title":"Exploratory Study of Privacy Preserving Fraud Detection","authors":"Rémi Canillas, Rania Talbi, S. Bouchenak, Omar Hasan, L. Brunie, Laurent Sarrat","doi":"10.1145/3284028.3284032","DOIUrl":"https://doi.org/10.1145/3284028.3284032","url":null,"abstract":"With the wide adoption of the Internet, digital transactions surge exponentially and so do the impersonation fraud. While machine learning techniques show strong promise to be the building block for digital fraud detection systems, clients may be reluctant to share the raw data with such systems due to privacy concerns. The emerging privacy preserving machine learning techniques that employ homomorphic encryption to resolve this conundrum unfortunately increases the computational overhead of detection. In this paper, we present a first-of-a-kind empirical performance study of a private fraud detection system developed at SiS ID, a French business security platform. A privacy-preserving decision tree which can classify transactions into four risk classes (safe, moderately risky, very risky and fraud) is trained on more than 160000 real world transactions, and we quantitatively compare the classification accuracy, latency and network bandwidth under various combinations of encryption parameters and learning hyper-parameters, in order to explore the impact of the configuration on the performances. Our results show that the computation and communication overhead of processing encrypted data increases by an order of magnitude of 5, and highly depends on the configuration of the encryption key and the number of nodes in the decision tree.","PeriodicalId":285212,"journal":{"name":"Proceedings of the 19th International Middleware Conference Industry","volume":"8 23","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120935887","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Emerging security solutions for cloud commonly operate in two phases, data collection and analytics. Data collection phase provides visibility into cloud resources (images, containers, VMs, etc.) and analytics derives insights built on data. Analytics phase is commonly decoupled from data collection and cloud resources as a separate, scalable pipeline. This enables cloud-scale operation via separation of concerns and overheads. Analytics focus on deriving high-value insights from data, and data collection focuses on efficient and minimally-intrusive inspection and introspection techniques. However, this model breaks traditional security solutions, such as endpoint managers, malware and compliance checkers, that are designed to run locally inside the systems they are securing. The common cloud strategy to address this problem has been to rewrite existing solutions to "work from data" instead of "working inside the system". This requires huge amount of resources and effort, and has fueled a slew of new "cloud-native security" solutions in the field. In this paper we approach this problem from a different angle. Instead of rewriting security solutions to work from data, we explore how to reuse existing security solutions as black-box analytics in the cloud. We present DéjàVu, a framework that makes data accessible to traditional software by mimicking a system veneer over the data. We achieve this by re-building a standard native POSIX system interface over the data. We enable traditional security applications to run unmodified in a black-box fashion. We validate our framework with state of the art third party security solutions and demonstrate that they can be operated with modest overhead.
{"title":"DéjàVu","authors":"S. Nadgowda, C. Isci, M. Bal","doi":"10.1145/3284028.3284031","DOIUrl":"https://doi.org/10.1145/3284028.3284031","url":null,"abstract":"Emerging security solutions for cloud commonly operate in two phases, data collection and analytics. Data collection phase provides visibility into cloud resources (images, containers, VMs, etc.) and analytics derives insights built on data. Analytics phase is commonly decoupled from data collection and cloud resources as a separate, scalable pipeline. This enables cloud-scale operation via separation of concerns and overheads. Analytics focus on deriving high-value insights from data, and data collection focuses on efficient and minimally-intrusive inspection and introspection techniques. However, this model breaks traditional security solutions, such as endpoint managers, malware and compliance checkers, that are designed to run locally inside the systems they are securing. The common cloud strategy to address this problem has been to rewrite existing solutions to \"work from data\" instead of \"working inside the system\". This requires huge amount of resources and effort, and has fueled a slew of new \"cloud-native security\" solutions in the field. In this paper we approach this problem from a different angle. Instead of rewriting security solutions to work from data, we explore how to reuse existing security solutions as black-box analytics in the cloud. We present DéjàVu, a framework that makes data accessible to traditional software by mimicking a system veneer over the data. We achieve this by re-building a standard native POSIX system interface over the data. We enable traditional security applications to run unmodified in a black-box fashion. We validate our framework with state of the art third party security solutions and demonstrate that they can be operated with modest overhead.","PeriodicalId":285212,"journal":{"name":"Proceedings of the 19th International Middleware Conference Industry","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116853940","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jie Li, Hai-Fei Liu, C. Gui, Jianyu Chen, Zhenyuan Ni, Ning Wang, Yuan Chen
We present the design and implementation of a visual search system for real time image retrieval on JD.com, the world's third largest and China's largest e-commerce site. We demonstrate that our system can support real time visual search with hundreds of billions of product images at sub-second timescales and handle frequent image updates through distributed hierarchical architecture and efficient indexing methods. We hope that sharing our practice with our real production system will inspire the middleware community's interest and appreciation for building practical large scale systems for emerging applications, such as e-commerce visual search.
{"title":"The Design and Implementation of a Real Time Visual Search System on JD E-commerce Platform","authors":"Jie Li, Hai-Fei Liu, C. Gui, Jianyu Chen, Zhenyuan Ni, Ning Wang, Yuan Chen","doi":"10.1145/3284028.3284030","DOIUrl":"https://doi.org/10.1145/3284028.3284030","url":null,"abstract":"We present the design and implementation of a visual search system for real time image retrieval on JD.com, the world's third largest and China's largest e-commerce site. We demonstrate that our system can support real time visual search with hundreds of billions of product images at sub-second timescales and handle frequent image updates through distributed hierarchical architecture and efficient indexing methods. We hope that sharing our practice with our real production system will inspire the middleware community's interest and appreciation for building practical large scale systems for emerging applications, such as e-commerce visual search.","PeriodicalId":285212,"journal":{"name":"Proceedings of the 19th International Middleware Conference Industry","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117160546","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kubernetes is a very popular and fast-growing container orchestration platform that automates the process of deploying and managing multi-container applications at scale. Users can specify required and maximum values of resources they need for their containers and Kubernetes realizes them by interfacing with lower levels (container runtime which in turn can use OS capabilities) of the stack for enforcing them. Kubernetes supports differentiated QoS classes - Guaranteed, Burstable, and Best-effort - in order of decreasing priority based on the resource size specifications for CPU and memory capacity. This allows many applications to obtain a desired level of QoS (performance isolation and throughput) when CPU or memory capacity management can provide them. However, when workloads may be critically dependent for their performance on another resource, namely network bandwidth, Kubernetes has no means to meet their QoS needs. Networking between pods in Kubernetes is supported with plug-ins and the network resource is not managed directly. In this work, we propose NBWGuard, a design for network bandwidth management and evaluate its implementation. NBWGuard lets Kubernetes manage network bandwidth as a resource (like CPU or memory capacity) while still using plug-ins for realizing the network specification desired by users. Consistent with Kubernetes approach to application QoS based on resource allocation NBWGuard also supports the 3 QoS classes: Guaranteed, Burstable, and Best-effort with respect to network bandwidth. NBWGuard is evaluated with iperf benchmark on real cloud environment, and the evaluation results demonstrate that it is able to provide network bandwidth isolation without impact on overall throughput.
{"title":"NBWGuard","authors":"Cong Xu, K. Rajamani, Wesley Felter","doi":"10.1145/3284028.3284033","DOIUrl":"https://doi.org/10.1145/3284028.3284033","url":null,"abstract":"Kubernetes is a very popular and fast-growing container orchestration platform that automates the process of deploying and managing multi-container applications at scale. Users can specify required and maximum values of resources they need for their containers and Kubernetes realizes them by interfacing with lower levels (container runtime which in turn can use OS capabilities) of the stack for enforcing them. Kubernetes supports differentiated QoS classes - Guaranteed, Burstable, and Best-effort - in order of decreasing priority based on the resource size specifications for CPU and memory capacity. This allows many applications to obtain a desired level of QoS (performance isolation and throughput) when CPU or memory capacity management can provide them. However, when workloads may be critically dependent for their performance on another resource, namely network bandwidth, Kubernetes has no means to meet their QoS needs. Networking between pods in Kubernetes is supported with plug-ins and the network resource is not managed directly. In this work, we propose NBWGuard, a design for network bandwidth management and evaluate its implementation. NBWGuard lets Kubernetes manage network bandwidth as a resource (like CPU or memory capacity) while still using plug-ins for realizing the network specification desired by users. Consistent with Kubernetes approach to application QoS based on resource allocation NBWGuard also supports the 3 QoS classes: Guaranteed, Burstable, and Best-effort with respect to network bandwidth. NBWGuard is evaluated with iperf benchmark on real cloud environment, and the evaluation results demonstrate that it is able to provide network bandwidth isolation without impact on overall throughput.","PeriodicalId":285212,"journal":{"name":"Proceedings of the 19th International Middleware Conference Industry","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127169841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Josep Sampé, G. Vernik, Marc Sánchez Artigas, P. López
Unexpectedly, the rise of serverless computing has also collaterally started the "democratization" of massive-scale data parallelism. This new trend heralded by PyWren pursues to enable untrained users to execute single-machine code in the cloud at massive scale through platforms like AWS Lambda. Inspired by this vision, this industry paper presents IBM-PyWren, which continues the pioneering work begun by PyWren in this field. It must be noted that IBM-PyWren is not, however, just a mere reimplementation of PyWren's API atop IBM Cloud Functions. Rather, it is must be viewed as an advanced extension of PyWren to run broader MapReduce jobs. We describe the design, innovative features (API extensions, data discovering & partitioning, composability, etc.) and performance of IBM-PyWren, along with the challenges encountered during its implementation.
{"title":"Serverless Data Analytics in the IBM Cloud","authors":"Josep Sampé, G. Vernik, Marc Sánchez Artigas, P. López","doi":"10.1145/3284028.3284029","DOIUrl":"https://doi.org/10.1145/3284028.3284029","url":null,"abstract":"Unexpectedly, the rise of serverless computing has also collaterally started the \"democratization\" of massive-scale data parallelism. This new trend heralded by PyWren pursues to enable untrained users to execute single-machine code in the cloud at massive scale through platforms like AWS Lambda. Inspired by this vision, this industry paper presents IBM-PyWren, which continues the pioneering work begun by PyWren in this field. It must be noted that IBM-PyWren is not, however, just a mere reimplementation of PyWren's API atop IBM Cloud Functions. Rather, it is must be viewed as an advanced extension of PyWren to run broader MapReduce jobs. We describe the design, innovative features (API extensions, data discovering & partitioning, composability, etc.) and performance of IBM-PyWren, along with the challenges encountered during its implementation.","PeriodicalId":285212,"journal":{"name":"Proceedings of the 19th International Middleware Conference Industry","volume":"30 5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125968174","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper introduces BcWAN, a roaming solution for an IoT LoRa-based network that allows IoT end-devices to deliver data to their home network going through foreign 1 gateways. Our architecture removes the central core network and replaces it with a blockchain that handles the network access control. Any gateway in the system can communicate directly with another gateway in a peer-to-peer manner while maintaining confidentiality, integrity and soundness. Our work solves the fair exchange problem introduced in such architecture where no third party is involved thanks to a combination of encryption and specific blockchain techniques like custom script operators. We implement a proof of concept of the BcWAN architecture to gather an insight of the performance of the solution. We outline that BcWAN itself does not add any significant overhead to a near real-time IoT application by presenting preliminary test results.
{"title":"BcWAN","authors":"Mehdi Bezahaf, Gaëtan Cathelain, Tony Ducrocq","doi":"10.1145/3284028.3284036","DOIUrl":"https://doi.org/10.1145/3284028.3284036","url":null,"abstract":"This paper introduces BcWAN, a roaming solution for an IoT LoRa-based network that allows IoT end-devices to deliver data to their home network going through foreign 1 gateways. Our architecture removes the central core network and replaces it with a blockchain that handles the network access control. Any gateway in the system can communicate directly with another gateway in a peer-to-peer manner while maintaining confidentiality, integrity and soundness. Our work solves the fair exchange problem introduced in such architecture where no third party is involved thanks to a combination of encryption and specific blockchain techniques like custom script operators. We implement a proof of concept of the BcWAN architecture to gather an insight of the performance of the solution. We outline that BcWAN itself does not add any significant overhead to a near real-time IoT application by presenting preliminary test results.","PeriodicalId":285212,"journal":{"name":"Proceedings of the 19th International Middleware Conference Industry","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131766733","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：