Wide-area Edge Database (WedgeDB) span globally and store data closer to the users. We term this the Global Edge Data management problem. In such an environment, aspects such as data storage, retrieval, transaction processing, and protection from malicious actors need to be addressed by any data management system that aims to consider itself a viable solution. Although blockchain technology (both permissioned and permissionless) has provided a way to address these concerns, transaction processing in these environments is still challenging. WedgeDB is an attempt to address these security problems in edge-cloud data systems [2]. The main goals of WedgeDB are to support distributed transaction processing coupled with secure transaction execution. Data stored in WedgeDB is partitioned into clusters with each cluster handling a unique set of keys. WedgeDB is a collection of such clusters. This is shown in figure 1. In figure 1, each cluster Cx stores a unique set of keys. This partitioning scheme allows us to build a distributed transaction model which runs transactions on a subset of clusters in the network. Transactions in WedgeDB are serializable. In WedgeDB, clients perform read operations as part of a transaction via read requests which can be sent to any of the WedgeDB nodes. The read operations are added to the transaction's history. Write operations are cached by the client until a commit is called which sends the transaction object containing the read history and write operations to a WedgeDB node to be committed. Transactions are processed in batches called Epochs. Each cluster maintains a leader which receives transactions and groups them into epochs. A cluster in WedgeDB contains 3f + 1 nodes (where f is the number of tolerable faulty nodes) and PBFT[1] is used to attain consensus among the nodes when executing transactions. Keys modified during an epoch are added to a Merkle tree which is used to verify changes to keys handled by the cluster. During transaction execution, proof of transaction execution is generated in the form of signed data blocks by the nodes in the cluster. At least f + 1 signed messages must be gathered before an epoch can be committed. These data blocks along with the root of the Merkle tree are stored in an SMR log where each entry in the SMR log corresponds to an epoch. Transactions that contain keys from different clusters are executed via two-phase commit. During the prepare phase a remote cluster executes PBFT within its local cluster and checks for dependency violations before moving ahead with the commit phase. Committed epoch may not have committed transactions and therefore an additional parameter is used to indicate the last committed epoch. This parameter combined with the dependency vector help in finding out serializability violations and abort transactions. With WedgeDB, transactions that affect only a few clusters do not require global consensus to commit. Transactions that read keys from a number of c
{"title":"WedgeDB","authors":"Abhishek A. Singh, Faisal Nawab","doi":"10.1145/3357223.3365444","DOIUrl":"https://doi.org/10.1145/3357223.3365444","url":null,"abstract":"Wide-area Edge Database (WedgeDB) span globally and store data closer to the users. We term this the Global Edge Data management problem. In such an environment, aspects such as data storage, retrieval, transaction processing, and protection from malicious actors need to be addressed by any data management system that aims to consider itself a viable solution. Although blockchain technology (both permissioned and permissionless) has provided a way to address these concerns, transaction processing in these environments is still challenging. WedgeDB is an attempt to address these security problems in edge-cloud data systems [2]. The main goals of WedgeDB are to support distributed transaction processing coupled with secure transaction execution. Data stored in WedgeDB is partitioned into clusters with each cluster handling a unique set of keys. WedgeDB is a collection of such clusters. This is shown in figure 1. In figure 1, each cluster Cx stores a unique set of keys. This partitioning scheme allows us to build a distributed transaction model which runs transactions on a subset of clusters in the network. Transactions in WedgeDB are serializable. In WedgeDB, clients perform read operations as part of a transaction via read requests which can be sent to any of the WedgeDB nodes. The read operations are added to the transaction's history. Write operations are cached by the client until a commit is called which sends the transaction object containing the read history and write operations to a WedgeDB node to be committed. Transactions are processed in batches called Epochs. Each cluster maintains a leader which receives transactions and groups them into epochs. A cluster in WedgeDB contains 3f + 1 nodes (where f is the number of tolerable faulty nodes) and PBFT[1] is used to attain consensus among the nodes when executing transactions. Keys modified during an epoch are added to a Merkle tree which is used to verify changes to keys handled by the cluster. During transaction execution, proof of transaction execution is generated in the form of signed data blocks by the nodes in the cluster. At least f + 1 signed messages must be gathered before an epoch can be committed. These data blocks along with the root of the Merkle tree are stored in an SMR log where each entry in the SMR log corresponds to an epoch. Transactions that contain keys from different clusters are executed via two-phase commit. During the prepare phase a remote cluster executes PBFT within its local cluster and checks for dependency violations before moving ahead with the commit phase. Committed epoch may not have committed transactions and therefore an additional parameter is used to indicate the last committed epoch. This parameter combined with the dependency vector help in finding out serializability violations and abort transactions. With WedgeDB, transactions that affect only a few clusters do not require global consensus to commit. Transactions that read keys from a number of c","PeriodicalId":91949,"journal":{"name":"Proceedings of the ... ACM Symposium on Cloud Computing [electronic resource] : SOCC ... ... SoCC (Conference)","volume":"13 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82473814","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The transition of traditional IT to cloud architectures and technologies is a trillion-dollar commercial opportunity, and we're still in the early innings of that shift. Venture-backed startups are competing alongside the big three platform providers to bring cloud technologies to market. What types of companies are VC-investable, what advantages do tiny teams have, and what does leading venture capital firm and cloud investor Greylock Partners look for? This talk will orient an academic audience in the considerations of early technology company-building, outline areas of investing interest, and discuss some common pitfalls for startups emerging from academia.
{"title":"Startups That Stand Out from the Cloud","authors":"Sarah Guo","doi":"10.1145/3357223.3365868","DOIUrl":"https://doi.org/10.1145/3357223.3365868","url":null,"abstract":"The transition of traditional IT to cloud architectures and technologies is a trillion-dollar commercial opportunity, and we're still in the early innings of that shift. Venture-backed startups are competing alongside the big three platform providers to bring cloud technologies to market. What types of companies are VC-investable, what advantages do tiny teams have, and what does leading venture capital firm and cloud investor Greylock Partners look for? This talk will orient an academic audience in the considerations of early technology company-building, outline areas of investing interest, and discuss some common pitfalls for startups emerging from academia.","PeriodicalId":91949,"journal":{"name":"Proceedings of the ... ACM Symposium on Cloud Computing [electronic resource] : SOCC ... ... SoCC (Conference)","volume":"18 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85347381","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kubernetes is an open-source cluster orchestration system for containerized workloads to reduce idiosyncrasy across cloud vendors [2]. Using Kubernetes, Cruise has built a multi-tenant platform with thousands of cores and tens of terabytes of memory. Such a scale is possible in part thanks to the declarative abstraction of Kubernetes, where desired states are described in YAML manifests [5]. However, YAML as a data serialization format is unfit for workload specification. Structured data in YAML are untyped and prone to wrong indents and missing fields. Due to poor meta-programming support, composing YAML with control logic---loops and branches---suffers from YAML fragmentation and indentation tracking (example at bit.ly/yml-hell). Moreover, YAML manifests are often generated by filling a shared template with cluster-specific parameters---the image tag and the replica count might differ in development and production environments. Existing templating tools---Helm [11], Kustomize [9], Kapitan [7] and the likes---assume these parameters are statically known and use CLIs to query dynamic ones, such as secrets stored in HashiCorp Vault [10]. Such scheme is hard to test, since side effects escape through CLIs, and highly depends on the execution environment, since CLI versions vary across machines or might not exist. Not least, YAML manifests describe the eventual state but not how existing workloads will be affected. Blindly applying the manifest---for example, from a stale version of code---can be disastrous and cause unexpected outages. Isopod presents an alternative configuration paradigm by treating Kubernetes objects as first-class citizens. Without intermediate YAML artifacts, Isopod renders Kubernetes objects directly in Protocol Buffers [8], so they are strongly typed and consumed directly by the Kubernetes API. With Isopod, configurations are scripted in Starlark [3], a Python dialect by Google also used by Bazel [1] and Buck [4] build systems. To replace CLI dependencies, Isopod extends Starlark with runtime built-ins to access services and utilities such as Vault, Kubernetes apiserver, Base64 encoder, and UUID generator, etc. Isopod uses a separate runtime for unit tests to mock all built-ins, providing test coverage that was not possible before. Isopod is also hermetic and secure. The common reliance on the kubeconfig file for cluster authentication leaks secrets to disk, a security risk if working from a shared host, such as a cluster node or CICD worker. Instead, Isopod builds Oauth2 tokens [6] to the target cluster using the Identity & Access Management (IAM) service of the cloud vendor. Application secrets are stored in Vault and queried at runtime. Hence, no secrets escape to the disk. In fact, Isopod prohibits disk IO except for loading Starlark modules from other scripts. No external libraries can be loaded unless explicitly implemented as an Isopod built-in. Distributed as a single binary, Isopod is self-contained with all dependen
{"title":"Isopod: An Expressive DSL for Kubernetes Configuration","authors":"Charles Xu, Dmitry Ilyevskiy","doi":"10.1145/3357223.3365759","DOIUrl":"https://doi.org/10.1145/3357223.3365759","url":null,"abstract":"Kubernetes is an open-source cluster orchestration system for containerized workloads to reduce idiosyncrasy across cloud vendors [2]. Using Kubernetes, Cruise has built a multi-tenant platform with thousands of cores and tens of terabytes of memory. Such a scale is possible in part thanks to the declarative abstraction of Kubernetes, where desired states are described in YAML manifests [5]. However, YAML as a data serialization format is unfit for workload specification. Structured data in YAML are untyped and prone to wrong indents and missing fields. Due to poor meta-programming support, composing YAML with control logic---loops and branches---suffers from YAML fragmentation and indentation tracking (example at bit.ly/yml-hell). Moreover, YAML manifests are often generated by filling a shared template with cluster-specific parameters---the image tag and the replica count might differ in development and production environments. Existing templating tools---Helm [11], Kustomize [9], Kapitan [7] and the likes---assume these parameters are statically known and use CLIs to query dynamic ones, such as secrets stored in HashiCorp Vault [10]. Such scheme is hard to test, since side effects escape through CLIs, and highly depends on the execution environment, since CLI versions vary across machines or might not exist. Not least, YAML manifests describe the eventual state but not how existing workloads will be affected. Blindly applying the manifest---for example, from a stale version of code---can be disastrous and cause unexpected outages. Isopod presents an alternative configuration paradigm by treating Kubernetes objects as first-class citizens. Without intermediate YAML artifacts, Isopod renders Kubernetes objects directly in Protocol Buffers [8], so they are strongly typed and consumed directly by the Kubernetes API. With Isopod, configurations are scripted in Starlark [3], a Python dialect by Google also used by Bazel [1] and Buck [4] build systems. To replace CLI dependencies, Isopod extends Starlark with runtime built-ins to access services and utilities such as Vault, Kubernetes apiserver, Base64 encoder, and UUID generator, etc. Isopod uses a separate runtime for unit tests to mock all built-ins, providing test coverage that was not possible before. Isopod is also hermetic and secure. The common reliance on the kubeconfig file for cluster authentication leaks secrets to disk, a security risk if working from a shared host, such as a cluster node or CICD worker. Instead, Isopod builds Oauth2 tokens [6] to the target cluster using the Identity & Access Management (IAM) service of the cloud vendor. Application secrets are stored in Vault and queried at runtime. Hence, no secrets escape to the disk. In fact, Isopod prohibits disk IO except for loading Starlark modules from other scripts. No external libraries can be loaded unless explicitly implemented as an Isopod built-in. Distributed as a single binary, Isopod is self-contained with all dependen","PeriodicalId":91949,"journal":{"name":"Proceedings of the ... ACM Symposium on Cloud Computing [electronic resource] : SOCC ... ... SoCC (Conference)","volume":"28 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76127529","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Although big data processing has become dramatically easier over the last decade, there has not been matching progress over big data debugging. It is estimated that users spend more than 50% of their time debugging their big data applications, wasting machine resources and taking longer to reach valuable insights. One cannot simply transplant traditional debugging techniques to big data. In this paper, we propose the TagSniff model, which can dramatically simplify data debugging for dataflows (the de-facto programming model for big data). It is based on two primitives -- tag and sniff -- that are flexible and expressive enough to model all common big data debugging scenarios. We then present Snoopy -- a general purpose monitoring and debugging system based on the TagSniff model. It supports both online and post-hoc debugging modes. Our experimental evaluation shows that Snoopy incurs a very low overhead on the main dataflow, 6% on average, as well as it is highly responsive to system events and users instructions.
{"title":"TagSniff","authors":"Bertty Contreras-Rojas, Jorge-Arnulfo Quiané-Ruiz, Zoi Kaoudi, Saravanan Thirumuruganathan","doi":"10.1145/3357223.3362738","DOIUrl":"https://doi.org/10.1145/3357223.3362738","url":null,"abstract":"Although big data processing has become dramatically easier over the last decade, there has not been matching progress over big data debugging. It is estimated that users spend more than 50% of their time debugging their big data applications, wasting machine resources and taking longer to reach valuable insights. One cannot simply transplant traditional debugging techniques to big data. In this paper, we propose the TagSniff model, which can dramatically simplify data debugging for dataflows (the de-facto programming model for big data). It is based on two primitives -- tag and sniff -- that are flexible and expressive enough to model all common big data debugging scenarios. We then present Snoopy -- a general purpose monitoring and debugging system based on the TagSniff model. It supports both online and post-hoc debugging modes. Our experimental evaluation shows that Snoopy incurs a very low overhead on the main dataflow, 6% on average, as well as it is highly responsive to system events and users instructions.","PeriodicalId":91949,"journal":{"name":"Proceedings of the ... ACM Symposium on Cloud Computing [electronic resource] : SOCC ... ... SoCC (Conference)","volume":"57 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76958713","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zhilin Zhang, Ke Wang, Weipeng Lin, A. Fu, R. C. Wong
As data outsourcing becomes popular, oblivious algorithms have raised extensive attentions. Their control flow and data access pattern appear to be independent of the input data they compute on. Oblivious algorithms, therefore, are especially suitable for secure processing in outsourced environments. In this work, we focus on oblivious shuffling algorithms that aim to shuffle encrypted data blocks outsourced to a cloud server without disclosing the actual permutation of blocks to the server. Existing oblivious shuffling algorithms suffer from issues of heavy communication cost and client computation cost for shuffling large-sized blocks because all outsourced blocks must be downloaded to the client for shuffling or peeling off extra encryption layers. To help eliminate this void, we introduce the "repeatable oblivious shuffling" notation that avoids moving blocks to the client and thus restricts the communication and client computation costs to be independent of the block size. For the first time, we present a concrete construction of repeatable oblivious shuffling using additively homomorphic encryption. The comprehensive evaluation of our construction shows its effective usability in practice for shuffling large-sized blocks.
{"title":"Repeatable Oblivious Shuffling of Large Outsourced Data Blocks","authors":"Zhilin Zhang, Ke Wang, Weipeng Lin, A. Fu, R. C. Wong","doi":"10.1145/3357223.3362732","DOIUrl":"https://doi.org/10.1145/3357223.3362732","url":null,"abstract":"As data outsourcing becomes popular, oblivious algorithms have raised extensive attentions. Their control flow and data access pattern appear to be independent of the input data they compute on. Oblivious algorithms, therefore, are especially suitable for secure processing in outsourced environments. In this work, we focus on oblivious shuffling algorithms that aim to shuffle encrypted data blocks outsourced to a cloud server without disclosing the actual permutation of blocks to the server. Existing oblivious shuffling algorithms suffer from issues of heavy communication cost and client computation cost for shuffling large-sized blocks because all outsourced blocks must be downloaded to the client for shuffling or peeling off extra encryption layers. To help eliminate this void, we introduce the \"repeatable oblivious shuffling\" notation that avoids moving blocks to the client and thus restricts the communication and client computation costs to be independent of the block size. For the first time, we present a concrete construction of repeatable oblivious shuffling using additively homomorphic encryption. The comprehensive evaluation of our construction shows its effective usability in practice for shuffling large-sized blocks.","PeriodicalId":91949,"journal":{"name":"Proceedings of the ... ACM Symposium on Cloud Computing [electronic resource] : SOCC ... ... SoCC (Conference)","volume":"18 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79682176","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Serverless computing is gaining popularity with the Function-asa-Service (FaaS) execution model. Without incurring overheads involved in provisioning cloud instances and with high availability and scalability, serverless computing allows developers to focus on implementation of core application logic using other well-developed cloud services. By abstracting the complex resource management task, serverless computing opens new opportunities for the cloud service adoption even to non-cloud experts [2]. With the popularity, many research results have been published using the FaaS execution model. They include investigation of serverless computing opportunities [1], proposing new serverless applications, function run-time optimization, and public service comparison. Without a common test benchmark suite, authors in the previous work had evaluated proposed systems using fairly simple FaaS applications, such as micro-benchmarks that emphasize specific resources exclusively, e.g., CPU, disk I/O, and network. However, such simple workloads do not represent realistic FaaS system applications, and the evaluations might not compare proposed systems appropriately. To overcome the limitation of lacking a comprehensive benchmark suite for the serverless computing and FaaS execution model, the authors create FunctionBench that provides various FaaS workloads that are ready to be executed on public cloud function execution services - AWS Lambda, Google Cloud Functions, and Azure functions1. Since the inception of serving the FaaS workloads, we keep working to expand the supported applications and add scenarios in big-data processing, back-end web applications, and security. To represent big-data applications, we add a MapReduce WordCount workload, which counts the number of occurrences of each word in a given partitioned input dataset from Wikipedia. To cover web back-end applications, we add Chameleon. The application renders a template using the Chameleon module in Python PIP library to create an HTML table of N rows and M columns that are provided as input arguments. Another web-related application is JSON serialize-deserialize module. The application performs JSON deserialization using a JSON-encoded string dataset (Awesome JSON Dataset) downloaded from a public object storage service, and it serializes the JSON object again. To represent security-related applications, we add Pyaes benchmark that performs private key-based encryption and decryption. It is a pure-Python implementation of the AES block-cipher algorithm in CTR mode. We also add gzip-compression benchmark to represent realistic disk IO-heavy applications. The degree (High, Medium, Low) of resource usage characteristics of newly proposed applications are summarized in Table 1. Please refer to [3] to read the description of comprehensive applications list. The proposed FunctionBench provides a variety of FaaS applications in multiple categories, and we are sure that it will enable fair evaluation o
{"title":"Practical Cloud Workloads for Serverless FaaS","authors":"Jeongchul Kim, Kyungyong Lee","doi":"10.1145/3357223.3365439","DOIUrl":"https://doi.org/10.1145/3357223.3365439","url":null,"abstract":"Serverless computing is gaining popularity with the Function-asa-Service (FaaS) execution model. Without incurring overheads involved in provisioning cloud instances and with high availability and scalability, serverless computing allows developers to focus on implementation of core application logic using other well-developed cloud services. By abstracting the complex resource management task, serverless computing opens new opportunities for the cloud service adoption even to non-cloud experts [2]. With the popularity, many research results have been published using the FaaS execution model. They include investigation of serverless computing opportunities [1], proposing new serverless applications, function run-time optimization, and public service comparison. Without a common test benchmark suite, authors in the previous work had evaluated proposed systems using fairly simple FaaS applications, such as micro-benchmarks that emphasize specific resources exclusively, e.g., CPU, disk I/O, and network. However, such simple workloads do not represent realistic FaaS system applications, and the evaluations might not compare proposed systems appropriately. To overcome the limitation of lacking a comprehensive benchmark suite for the serverless computing and FaaS execution model, the authors create FunctionBench that provides various FaaS workloads that are ready to be executed on public cloud function execution services - AWS Lambda, Google Cloud Functions, and Azure functions1. Since the inception of serving the FaaS workloads, we keep working to expand the supported applications and add scenarios in big-data processing, back-end web applications, and security. To represent big-data applications, we add a MapReduce WordCount workload, which counts the number of occurrences of each word in a given partitioned input dataset from Wikipedia. To cover web back-end applications, we add Chameleon. The application renders a template using the Chameleon module in Python PIP library to create an HTML table of N rows and M columns that are provided as input arguments. Another web-related application is JSON serialize-deserialize module. The application performs JSON deserialization using a JSON-encoded string dataset (Awesome JSON Dataset) downloaded from a public object storage service, and it serializes the JSON object again. To represent security-related applications, we add Pyaes benchmark that performs private key-based encryption and decryption. It is a pure-Python implementation of the AES block-cipher algorithm in CTR mode. We also add gzip-compression benchmark to represent realistic disk IO-heavy applications. The degree (High, Medium, Low) of resource usage characteristics of newly proposed applications are summarized in Table 1. Please refer to [3] to read the description of comprehensive applications list. The proposed FunctionBench provides a variety of FaaS applications in multiple categories, and we are sure that it will enable fair evaluation o","PeriodicalId":91949,"journal":{"name":"Proceedings of the ... ACM Symposium on Cloud Computing [electronic resource] : SOCC ... ... SoCC (Conference)","volume":"255 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79508562","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kunal Lillaney, Vasily Tarasov, David A. Pease, Randal C. Burns
Object storage is a low-cost, scalable component of cloud ecosystems. However, interface incompatibilities and performance limitations inhibit its adoption for emerging cloud-based workloads. Users are compelled to either run their applications over expensive block storage-based file systems or use inefficient file connectors over object stores. Dual access, the ability to read and write the same data through file systems interfaces and object storage APIs, has promise to improve performance and eliminate storage sprawl. We design and implement Agni1, an efficient, distributed, dual-access object storage file system (OSFS), that uses standard object storage APIs and cloud microservices. Our system overcomes the performance shortcomings of existing approaches by implementing a multi-tier write aggregating data structure and by integrating with existing cloud-native services. Moreover, Agni provides distributed access and a coherent namespace. Our experiments demonstrate that for representative workloads Agni improves performance by 20%--60% when compared with existing approaches.
{"title":"Agni","authors":"Kunal Lillaney, Vasily Tarasov, David A. Pease, Randal C. Burns","doi":"10.1145/3357223.3362703","DOIUrl":"https://doi.org/10.1145/3357223.3362703","url":null,"abstract":"Object storage is a low-cost, scalable component of cloud ecosystems. However, interface incompatibilities and performance limitations inhibit its adoption for emerging cloud-based workloads. Users are compelled to either run their applications over expensive block storage-based file systems or use inefficient file connectors over object stores. Dual access, the ability to read and write the same data through file systems interfaces and object storage APIs, has promise to improve performance and eliminate storage sprawl. We design and implement Agni1, an efficient, distributed, dual-access object storage file system (OSFS), that uses standard object storage APIs and cloud microservices. Our system overcomes the performance shortcomings of existing approaches by implementing a multi-tier write aggregating data structure and by integrating with existing cloud-native services. Moreover, Agni provides distributed access and a coherent namespace. Our experiments demonstrate that for representative workloads Agni improves performance by 20%--60% when compared with existing approaches.","PeriodicalId":91949,"journal":{"name":"Proceedings of the ... ACM Symposium on Cloud Computing [electronic resource] : SOCC ... ... SoCC (Conference)","volume":"28 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"78173490","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Over the last few years, we have experienced a massive transformation of the Software Defined Networking ecosystem with the development of SDNEnhancements, e.g., Statesman, ESPRES, Pane, and Pyretic, to provide better composability, better utilization of TCAM, consistent network updates, or congestion free updates. The end-result of this organic evolution is a disconnect between the SDN applications and the data-plane. A disconnect which can impact an SDN application's performance and efficacy. In this paper, we present the first systematic study of the interactions between SDNEnhancements and SDN applications -- we show that an SDN application's performance can be significantly impacted by these SDNEnhancements: for example, we observed that the efficiency of a traffic engineering SDN application was reduced by 24.8%. Motivated by these insights, we present, Mozart, a redesigned SDN controller centered around mitigating and reducing the impact of these SDNEnhancements. Using two prototypes interoperating with seven SDN applications and two SDNEnhancements, we demonstrate that our abstractions require minimal changes and can restore an SDN application's performance. We analyzed Mozart's scalability and overhead using large scale simulations of modern cloud networks and observed them to be negligible.
{"title":"Composing SDN Controller Enhancements with Mozart","authors":"Zhenyu Zhou, Theophilus A. Benson","doi":"10.1145/3357223.3362712","DOIUrl":"https://doi.org/10.1145/3357223.3362712","url":null,"abstract":"Over the last few years, we have experienced a massive transformation of the Software Defined Networking ecosystem with the development of SDNEnhancements, e.g., Statesman, ESPRES, Pane, and Pyretic, to provide better composability, better utilization of TCAM, consistent network updates, or congestion free updates. The end-result of this organic evolution is a disconnect between the SDN applications and the data-plane. A disconnect which can impact an SDN application's performance and efficacy. In this paper, we present the first systematic study of the interactions between SDNEnhancements and SDN applications -- we show that an SDN application's performance can be significantly impacted by these SDNEnhancements: for example, we observed that the efficiency of a traffic engineering SDN application was reduced by 24.8%. Motivated by these insights, we present, Mozart, a redesigned SDN controller centered around mitigating and reducing the impact of these SDNEnhancements. Using two prototypes interoperating with seven SDN applications and two SDNEnhancements, we demonstrate that our abstractions require minimal changes and can restore an SDN application's performance. We analyzed Mozart's scalability and overhead using large scale simulations of modern cloud networks and observed them to be negligible.","PeriodicalId":91949,"journal":{"name":"Proceedings of the ... ACM Symposium on Cloud Computing [electronic resource] : SOCC ... ... SoCC (Conference)","volume":"540 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87917602","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
H. Jeong, C. Shin, K. Shin, Hyeon-Jae Lee, Soo-Mook Moon
Future mobile applications, such as mobile cloud gaming or augmented reality, require not only high computation power but strict latency constraints. To provide computing resources with ultra-low latency, a new form of cloud infrastructure called edge cloud has been proposed, which distributes computing servers at the edges of the network. A primary concern of edge cloud is that a physical server running a service can change as the client moves, so the service has to be quickly migrated between servers for seamless computation offloading. This paper tackles the issue in the context of web applications, whose computation-intensive codes are written in JavaScript and webassembly. The basic building block of our system is a mobile web worker, which extends HTML5 web worker to support migration across the client, edge, and cloud servers. Our system migrates a mobile web worker from mobile device to an edge server to minimize execution latency. The immigrated worker can move again to other servers for better performance or service recovery. To implement the runtime migration of the worker, we use a novel serialization algorithm that captures the web worker state where webassembly functions and JavaScript objects are intermingled. Experimental result showed that our system could successfully migrate a non-trivial web worker running webassembly-version OpenCV within a few seconds, and achieved up to 8.4x speedup compared to offloading of pure JavaScript.
未来的移动应用程序，如移动云游戏或增强现实，不仅需要高计算能力，而且需要严格的延迟限制。为了提供超低延迟的计算资源，提出了一种新的云基础设施形式——边缘云，它将计算服务器分布在网络的边缘。边缘云的一个主要问题是，运行服务的物理服务器可能会随着客户端的移动而变化，因此服务必须在服务器之间快速迁移，以实现无缝的计算卸载。本文在web应用程序的背景下解决这个问题，这些应用程序的计算密集型代码是用JavaScript和webassembly编写的。我们系统的基本构建块是一个移动web worker，它扩展了HTML5 web worker以支持跨客户端、边缘和云服务器的迁移。我们的系统将移动web工作者从移动设备迁移到边缘服务器，以最小化执行延迟。迁移的工作人员可以再次移动到其他服务器以获得更好的性能或服务恢复。为了实现worker的运行时迁移，我们使用了一种新的序列化算法来捕获webassembly函数和JavaScript对象混合在一起的web worker状态。实验结果表明，我们的系统可以在几秒钟内成功迁移一个运行webassembly-version OpenCV的非平凡web worker，并且与纯JavaScript卸载相比，实现了高达8.4倍的加速。
{"title":"Seamless Offloading of Web App Computations From Mobile Device to Edge Clouds via HTML5 Web Worker Migration","authors":"H. Jeong, C. Shin, K. Shin, Hyeon-Jae Lee, Soo-Mook Moon","doi":"10.1145/3357223.3362735","DOIUrl":"https://doi.org/10.1145/3357223.3362735","url":null,"abstract":"Future mobile applications, such as mobile cloud gaming or augmented reality, require not only high computation power but strict latency constraints. To provide computing resources with ultra-low latency, a new form of cloud infrastructure called edge cloud has been proposed, which distributes computing servers at the edges of the network. A primary concern of edge cloud is that a physical server running a service can change as the client moves, so the service has to be quickly migrated between servers for seamless computation offloading. This paper tackles the issue in the context of web applications, whose computation-intensive codes are written in JavaScript and webassembly. The basic building block of our system is a mobile web worker, which extends HTML5 web worker to support migration across the client, edge, and cloud servers. Our system migrates a mobile web worker from mobile device to an edge server to minimize execution latency. The immigrated worker can move again to other servers for better performance or service recovery. To implement the runtime migration of the worker, we use a novel serialization algorithm that captures the web worker state where webassembly functions and JavaScript objects are intermingled. Experimental result showed that our system could successfully migrate a non-trivial web worker running webassembly-version OpenCV within a few seconds, and achieved up to 8.4x speedup compared to offloading of pure JavaScript.","PeriodicalId":91949,"journal":{"name":"Proceedings of the ... ACM Symposium on Cloud Computing [electronic resource] : SOCC ... ... SoCC (Conference)","volume":"64 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75214399","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Despite extensive investigation of job scheduling in data-intensive computation frameworks, less consideration has been given to optimizing job partitioning for resource utilization and efficient processing. Instead, partitioning and job sizing are a form of dark art, typically left to developer intuition and trial-and-error style experimentation. In this work, we propose that just as job scheduling and resource allocation are out-sourced to a trusted mechanism external to the workload, so too should be the responsibility for partitioning data as a determinant for task size. Job partitioning essentially involves determining the partition sizes to match the resource allocation at the finest granularity. This is a complex, multi-dimensional problem that is highly application specific: resource allocation, computational runtime, shuffle and reduce communication requirements, and task startup overheads all have strong influence on the most effective task size for efficient processing. Depending on the partition size, the job completion time can differ by as much as 10 times! Fortunately, we observe a general trend underlying the tradeoff between full resource utilization and system overhead across different settings. The optimal job partition size balances these two conflicting forces. Given this trend, we design Libra to automate job partitioning as a framework extension. We integrate Libra with Spark and evaluate its performance on EC2. Compared to state-of-the-art techniques, Libra can reduce the individual job execution time by 25% to 70%.
{"title":"Libra and the Art of Task Sizing in Big-Data Analytic Systems","authors":"Ruikang Li, Peizhen Guo, Bo Hu, Wenjun Hu","doi":"10.1145/3357223.3362720","DOIUrl":"https://doi.org/10.1145/3357223.3362720","url":null,"abstract":"Despite extensive investigation of job scheduling in data-intensive computation frameworks, less consideration has been given to optimizing job partitioning for resource utilization and efficient processing. Instead, partitioning and job sizing are a form of dark art, typically left to developer intuition and trial-and-error style experimentation. In this work, we propose that just as job scheduling and resource allocation are out-sourced to a trusted mechanism external to the workload, so too should be the responsibility for partitioning data as a determinant for task size. Job partitioning essentially involves determining the partition sizes to match the resource allocation at the finest granularity. This is a complex, multi-dimensional problem that is highly application specific: resource allocation, computational runtime, shuffle and reduce communication requirements, and task startup overheads all have strong influence on the most effective task size for efficient processing. Depending on the partition size, the job completion time can differ by as much as 10 times! Fortunately, we observe a general trend underlying the tradeoff between full resource utilization and system overhead across different settings. The optimal job partition size balances these two conflicting forces. Given this trend, we design Libra to automate job partitioning as a framework extension. We integrate Libra with Spark and evaluate its performance on EC2. Compared to state-of-the-art techniques, Libra can reduce the individual job execution time by 25% to 70%.","PeriodicalId":91949,"journal":{"name":"Proceedings of the ... ACM Symposium on Cloud Computing [electronic resource] : SOCC ... ... SoCC (Conference)","volume":"19 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75929075","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：