Kwangsung Oh, A. Raghavan, A. Chandra, J. Weissman
Many Cloud applications exploit the diversity of storage options in a data center to achieve desired cost, performance, and durability tradeoffs. It is common to see applications using a combination of memory, local disk, and archival storage tiers within a single data center to meet their needs. For example, hot data can be kept in memory using ElastiCache, and colder data in cheaper, slower storage such as S3, using Amazon as an example. For user-facing applications, a recent trend is to exploit multiple data centers for data placement to enable better latency of access from users to their data. The conventional wisdom is that co-location of computation and storage within the same data center is a key to application performance, so that applications running within a data center are often still limited to access local data. In this paper, using experiments on Amazon, Microsoft, and Google clouds, we show that this assumption is false, and that accessing data in nearby data centers may be faster than local access at different or even same points in the storage hierarchy. This can lead to not only better performance, but also reduced cost, simpler consistency policies and reconsidering data locality in multiple DCs environment. This argues for an expansion of cloud storage tiers to consider non-local storage options, and has interesting implications for the design of a distributed storage system.
{"title":"Redefining Data Locality for Cross-Data Center Storage","authors":"Kwangsung Oh, A. Raghavan, A. Chandra, J. Weissman","doi":"10.1145/2756594.2756596","DOIUrl":"https://doi.org/10.1145/2756594.2756596","url":null,"abstract":"Many Cloud applications exploit the diversity of storage options in a data center to achieve desired cost, performance, and durability tradeoffs. It is common to see applications using a combination of memory, local disk, and archival storage tiers within a single data center to meet their needs. For example, hot data can be kept in memory using ElastiCache, and colder data in cheaper, slower storage such as S3, using Amazon as an example. For user-facing applications, a recent trend is to exploit multiple data centers for data placement to enable better latency of access from users to their data. The conventional wisdom is that co-location of computation and storage within the same data center is a key to application performance, so that applications running within a data center are often still limited to access local data. In this paper, using experiments on Amazon, Microsoft, and Google clouds, we show that this assumption is false, and that accessing data in nearby data centers may be faster than local access at different or even same points in the storage hierarchy. This can lead to not only better performance, but also reduced cost, simpler consistency policies and reconsidering data locality in multiple DCs environment. This argues for an expansion of cloud storage tiers to consider non-local storage options, and has interesting implications for the design of a distributed storage system.","PeriodicalId":283088,"journal":{"name":"Proceedings of the 2nd International Workshop on Software-Defined Ecosystems","volume":"105 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133309923","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Paula Austel, Han Chen, Thomas A. Mikalsen, I. Rouvellou, Upendra Sharma, I. Silva-Lepe, R. Subramanian
To help drive top line growth of their businesses, the development and IT organizations are under increasing pressure to create and deliver applications at ever faster paces. The advent of Cloud Computing has not only lowered the cost of IT operations but also enabled the notion of continuous delivery, which promises to radically reduce frictions in DevOps processes and speed up the product delivery cycle. With increased demand on functionality and feature, we have also seen these applications becoming more sophisticated, often integrating multiple modern programming models and techniques with the traditional n-tier web application into a composite application. This paper proposes an architectural blueprint for improved continuous delivery of these complex composite applications. It treats a solution as a holistic entity comprised of application logic and software-defined environment that the logic relies on. It also proposes a collaborative approach to software-defined Platform-as-a-Service environment building. This being an ongoing research project, this paper also briefly describes prototype, work-in-progress and thoughts on future directions.
{"title":"Continuous Delivery of Composite Solutions: A Case for Collaborative Software Defined PaaS Environments","authors":"Paula Austel, Han Chen, Thomas A. Mikalsen, I. Rouvellou, Upendra Sharma, I. Silva-Lepe, R. Subramanian","doi":"10.1145/2756594.2756595","DOIUrl":"https://doi.org/10.1145/2756594.2756595","url":null,"abstract":"To help drive top line growth of their businesses, the development and IT organizations are under increasing pressure to create and deliver applications at ever faster paces. The advent of Cloud Computing has not only lowered the cost of IT operations but also enabled the notion of continuous delivery, which promises to radically reduce frictions in DevOps processes and speed up the product delivery cycle. With increased demand on functionality and feature, we have also seen these applications becoming more sophisticated, often integrating multiple modern programming models and techniques with the traditional n-tier web application into a composite application. This paper proposes an architectural blueprint for improved continuous delivery of these complex composite applications. It treats a solution as a holistic entity comprised of application logic and software-defined environment that the logic relies on. It also proposes a collaborative approach to software-defined Platform-as-a-Service environment building. This being an ongoing research project, this paper also briefly describes prototype, work-in-progress and thoughts on future directions.","PeriodicalId":283088,"journal":{"name":"Proceedings of the 2nd International Workshop on Software-Defined Ecosystems","volume":"45 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121312591","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
L. Peterson, Scott Baker, M. D. Leenheer, A. Bavier, S. Bhatia, Mike Wawrzoniak, Jude C. Nelson, J. Hartman
This paper describes XOS, a cloud operating system designed to manage hardware and software resources across a multi-tiered cloud. XOS raises the level of abstraction in an IaaS cloud architecture by elevating scalable software services to first-class objects. This involves adopting three design principles: (1) Everything-as-a-Service (XaaS) (services are building blocks, and combinations of those building blocks are also services); (2) Multi-tenancy (a tenant relationship links one service to another, and facilitates reasoning about safety, privacy and efficiency); and (3) Control/Data-plane separation (services are configured through a logically centralized service controller interface, but the controller is not on the data path between services). XOS applies these principles through the lens of an operating system - it defines a set of abstractions that support constructing multi-tenant services that can be folded back into XOS as available building blocks, while also extending the capabilities of conventional IaaS. The paper shows how these abstractions can be used to build a functional, evolvable, service-oriented cloud.
{"title":"XoS: An Extensible Cloud Operating System","authors":"L. Peterson, Scott Baker, M. D. Leenheer, A. Bavier, S. Bhatia, Mike Wawrzoniak, Jude C. Nelson, J. Hartman","doi":"10.1145/2756594.2756598","DOIUrl":"https://doi.org/10.1145/2756594.2756598","url":null,"abstract":"This paper describes XOS, a cloud operating system designed to manage hardware and software resources across a multi-tiered cloud. XOS raises the level of abstraction in an IaaS cloud architecture by elevating scalable software services to first-class objects. This involves adopting three design principles: (1) Everything-as-a-Service (XaaS) (services are building blocks, and combinations of those building blocks are also services); (2) Multi-tenancy (a tenant relationship links one service to another, and facilitates reasoning about safety, privacy and efficiency); and (3) Control/Data-plane separation (services are configured through a logically centralized service controller interface, but the controller is not on the data path between services). XOS applies these principles through the lens of an operating system - it defines a set of abstractions that support constructing multi-tenant services that can be folded back into XOS as available building blocks, while also extending the capabilities of conventional IaaS. The paper shows how these abstractions can be used to build a functional, evolvable, service-oriented cloud.","PeriodicalId":283088,"journal":{"name":"Proceedings of the 2nd International Workshop on Software-Defined Ecosystems","volume":"172 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115196105","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The distance between compute and data continues to grow and despite being declared "dead" a decade ago, the interfaces of the serial world (Posix) are still with us. Is it time to move on and to what? How can we give applications the desired flexibility, consistency and volume of data in large systems composed of thousands or millions of failing parts? The group Brent runs at Intel, the High Performance Data Division, has been shepherding the Lustre file system while working on the riddle of "what now" for big systems when it comes to feeding them data.
计算和数据之间的距离继续增长，尽管十年前被宣布“死亡”，串行世界的接口(Posix)仍然与我们同在。是时候向前看了吗?我们如何在由数千或数百万个故障部件组成的大型系统中为应用程序提供所需的灵活性、一致性和数据量?布伦特在英特尔(Intel)负责的高性能数据部门(High Performance Data Division)一直在管理Lustre文件系统，同时研究在向大系统提供数据时“现在该怎么办”的难题。
{"title":"Feeding the Beast: Getting Data into Big Systems","authors":"B. Gorda","doi":"10.1145/2756594.2756600","DOIUrl":"https://doi.org/10.1145/2756594.2756600","url":null,"abstract":"The distance between compute and data continues to grow and despite being declared \"dead\" a decade ago, the interfaces of the serial world (Posix) are still with us. Is it time to move on and to what? How can we give applications the desired flexibility, consistency and volume of data in large systems composed of thousands or millions of failing parts? The group Brent runs at Intel, the High Performance Data Division, has been shepherding the Lustre file system while working on the riddle of \"what now\" for big systems when it comes to feeding them data.","PeriodicalId":283088,"journal":{"name":"Proceedings of the 2nd International Workshop on Software-Defined Ecosystems","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114100368","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The fast evolving of data center virtualization has led to high volumes of network fabric traffic. While data center hosts multiple VDCs and network fabrics are managed as a single logical network entity, the network is practically unable to prevent a traffic flood in one VDC from affecting the other VDCs around it. Hence, the data center's network fabric becomes a focal point. Tailoring network to implement virtual fabrics to give each VDC the illusion that all the resources assigned to it are dedicated and isolated, just as if each VDC was connected by a separate physical network, is challenging. Realizing this vision concretely translates into building a network fabric that is virtualized, isolated and that prevents packet forwarding degradation. One of the significant challenges is then around resource constraints. In this paper we present a decentralized control for vFabrics based on a three-level control strategy that dynamically adjusts resource allocation over different time scales to fit vFabrics needs in terms of packet processing. Results show that control strategy guarantees a high fairness level among vFabrics, preserves high performance packet processing by maintaining processing latency stability, and optimizes resource utilisation.
{"title":"Virtual Fabric-based Approach for Virtual Data Center Network","authors":"Khalil Blaiech, O. Cherkaoui","doi":"10.1145/2756594.2756599","DOIUrl":"https://doi.org/10.1145/2756594.2756599","url":null,"abstract":"The fast evolving of data center virtualization has led to high volumes of network fabric traffic. While data center hosts multiple VDCs and network fabrics are managed as a single logical network entity, the network is practically unable to prevent a traffic flood in one VDC from affecting the other VDCs around it. Hence, the data center's network fabric becomes a focal point. Tailoring network to implement virtual fabrics to give each VDC the illusion that all the resources assigned to it are dedicated and isolated, just as if each VDC was connected by a separate physical network, is challenging. Realizing this vision concretely translates into building a network fabric that is virtualized, isolated and that prevents packet forwarding degradation. One of the significant challenges is then around resource constraints. In this paper we present a decentralized control for vFabrics based on a three-level control strategy that dynamically adjusts resource allocation over different time scales to fit vFabrics needs in terms of packet processing. Results show that control strategy guarantees a high fairness level among vFabrics, preserves high performance packet processing by maintaining processing latency stability, and optimizes resource utilisation.","PeriodicalId":283088,"journal":{"name":"Proceedings of the 2nd International Workshop on Software-Defined Ecosystems","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127050380","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Moustafa AbdelBaky, J. Montes, Mengsong Zou, M. Parashar
Federated computing has been shown to be an effective model for harnessing the capabilities and capacities of geographically- distributed resources in order to solve large science and en- gineering problems. However, traditional High Performance Computing (HPC) based federation models can be restrictive as they present users with a pre-defined set of resources and do not allow federations to evolve in response to changing resources or application needs. As emerging application workflows and the underlying resources become increasingly dynamic and exhibit changing requirements and constraints, they cannot be easily supported by such federation models. Instead, new federation models that are capable of dynamically adapting to these emerging needs are required. In this paper, we present a programmable dynamic federation model that uses software-defined environment concepts to drive the federation process and seamlessly adapt resource compositions at runtime. The resulting software-defined federation adapts to meet both requirements and constraints set by the user, application, and/or resource providers. In this paper we present the design and prototype implementation of such software-defined federation model, and demonstrate its operation and performance through a use case where heterogeneous, geographically distributed resources are federated based on user specifications, and the federation evolves over time following the requirements and constraints defined by the user.
{"title":"A Framework for Realizing Software-Defined Federations for Scientific Workflows","authors":"Moustafa AbdelBaky, J. Montes, Mengsong Zou, M. Parashar","doi":"10.1145/2756594.2756597","DOIUrl":"https://doi.org/10.1145/2756594.2756597","url":null,"abstract":"Federated computing has been shown to be an effective model for harnessing the capabilities and capacities of geographically- distributed resources in order to solve large science and en- gineering problems. However, traditional High Performance Computing (HPC) based federation models can be restrictive as they present users with a pre-defined set of resources and do not allow federations to evolve in response to changing resources or application needs. As emerging application workflows and the underlying resources become increasingly dynamic and exhibit changing requirements and constraints, they cannot be easily supported by such federation models. Instead, new federation models that are capable of dynamically adapting to these emerging needs are required. In this paper, we present a programmable dynamic federation model that uses software-defined environment concepts to drive the federation process and seamlessly adapt resource compositions at runtime. The resulting software-defined federation adapts to meet both requirements and constraints set by the user, application, and/or resource providers. In this paper we present the design and prototype implementation of such software-defined federation model, and demonstrate its operation and performance through a use case where heterogeneous, geographically distributed resources are federated based on user specifications, and the federation evolves over time following the requirements and constraints defined by the user.","PeriodicalId":283088,"journal":{"name":"Proceedings of the 2nd International Workshop on Software-Defined Ecosystems","volume":"74 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132034488","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Proceedings of the 2nd International Workshop on Software-Defined Ecosystems","authors":"","doi":"10.1145/2756594","DOIUrl":"https://doi.org/10.1145/2756594","url":null,"abstract":"","PeriodicalId":283088,"journal":{"name":"Proceedings of the 2nd International Workshop on Software-Defined Ecosystems","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126435365","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：