Pub Date : 2011-07-04DOI: 10.1109/SERVICES.2011.56
M. Wahib, Asim Munawar, M. Munetomo, K. Akama
Cloud computing is impacting the modern Internet computing and businesses in every aspect. One feature of clouds is the convenience of using the services offered by the cloud. Consequently, most cloud service providers use WS for users and developers to interface with the cloud. However, the current cloud WS are focused into core and fundamental modern computing functionalities. We anticipate as cloud developments tools mature and cloud applications become more popular, there will be an opportunity for designing and implementing applications/services to be embedded in the cloud for use by applications in the cloud. We propose a framework for WS deployment in the cloud to be usable by applications residing in the same cloud. The framework capitalizes on the cloud strong points to offer a higher value to the service consumer inside the cloud. The authoritative nature of clouds would enable more efficient models for WS publishing, indexing and description. Moreover, being hosted in the cloud, WScan build on the high scalability offered by the cloud with a much higher reliability. Finally, scheduling the instances using the WS in bundle with the WS instances could offer a LAN-like connectivity performance driving down the latency to the magnitude of lower microseconds. In this paper, we highlight the challenges and opportunities of cloud applications using cloud embedded Web services. We give a description of the different aspects by illustrating the different components, together with an end-to-end use case to show the applicability of the proposed system.
云计算正在影响着现代互联网计算和商业的各个方面。云的一个特性是使用云提供的服务的便利性。因此，大多数云服务提供商使用WS让用户和开发人员与云进行交互。然而，当前的云WS集中在核心和基本的现代计算功能上。我们预计，随着云开发工具的成熟和云应用程序的普及，将有机会设计和实现嵌入云中的应用程序/服务，供云中的应用程序使用。我们提出了一个用于在云中部署WS的框架，以供驻留在同一云中的应用程序使用。该框架利用了云的优点，为云中的服务消费者提供了更高的价值。云的权威性将为WS的发布、索引和描述提供更有效的模型。此外，由于托管在云中，WScan构建在云提供的高可伸缩性上，具有更高的可靠性。最后，使用WS in bundle与WS实例一起调度实例可以提供类似lan的连接性能，将延迟降低到更低的微秒级。在本文中，我们强调了使用云嵌入式Web服务的云应用程序的挑战和机遇。我们通过说明不同的组件给出了不同方面的描述，以及端到端用例来显示所建议系统的适用性。
{"title":"A Framework for Cloud Embedded Web Services Utilized by Cloud Applications","authors":"M. Wahib, Asim Munawar, M. Munetomo, K. Akama","doi":"10.1109/SERVICES.2011.56","DOIUrl":"https://doi.org/10.1109/SERVICES.2011.56","url":null,"abstract":"Cloud computing is impacting the modern Internet computing and businesses in every aspect. One feature of clouds is the convenience of using the services offered by the cloud. Consequently, most cloud service providers use WS for users and developers to interface with the cloud. However, the current cloud WS are focused into core and fundamental modern computing functionalities. We anticipate as cloud developments tools mature and cloud applications become more popular, there will be an opportunity for designing and implementing applications/services to be embedded in the cloud for use by applications in the cloud. We propose a framework for WS deployment in the cloud to be usable by applications residing in the same cloud. The framework capitalizes on the cloud strong points to offer a higher value to the service consumer inside the cloud. The authoritative nature of clouds would enable more efficient models for WS publishing, indexing and description. Moreover, being hosted in the cloud, WScan build on the high scalability offered by the cloud with a much higher reliability. Finally, scheduling the instances using the WS in bundle with the WS instances could offer a LAN-like connectivity performance driving down the latency to the magnitude of lower microseconds. In this paper, we highlight the challenges and opportunities of cloud applications using cloud embedded Web services. We give a description of the different aspects by illustrating the different components, together with an end-to-end use case to show the applicability of the proposed system.","PeriodicalId":429726,"journal":{"name":"2011 IEEE World Congress on Services","volume":"122 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114987802","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-07-04DOI: 10.1109/SERVICES.2011.73
B. Liver, K. Tice
The Service-Oriented Architecture (SOA) style is instrumental for Credit Suisse to manage the complexity of its very large application landscape. This paper briefly outlines how the Enterprise Integration Architecture governs the service landscape and continuously develops the Service-Oriented Architecture and its supporting middleware and tools. A brief summary of the current state of SOA at Credit Suisse is provided, which includes examples illustrating the necessity of strong and continuous governance and which sketches out some key topics for further development.
{"title":"SOA Service Design and Governance: Experience at Credit Suisse","authors":"B. Liver, K. Tice","doi":"10.1109/SERVICES.2011.73","DOIUrl":"https://doi.org/10.1109/SERVICES.2011.73","url":null,"abstract":"The Service-Oriented Architecture (SOA) style is instrumental for Credit Suisse to manage the complexity of its very large application landscape. This paper briefly outlines how the Enterprise Integration Architecture governs the service landscape and continuously develops the Service-Oriented Architecture and its supporting middleware and tools. A brief summary of the current state of SOA at Credit Suisse is provided, which includes examples illustrating the necessity of strong and continuous governance and which sketches out some key topics for further development.","PeriodicalId":429726,"journal":{"name":"2011 IEEE World Congress on Services","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121088820","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-07-04DOI: 10.1109/SERVICES.2011.14
Hemant Shah, Ganesh Krishnan, P. Williams, A. Vogler, R. D. Allard, P. Nadkarni
Electronic medical record systems (EMRs) can be made more attractive to the clinicians if Clinical Decision Support Systems (CDSS) are integrated with them. However, CDSS have to be developed with integration in mind, such that they may be integrated not just with the local EMR but EMRs developed by others as well. Web Services Technology ameliorates the challenge of integration if the CDSS is well-designed but several other issues still need to be considered. The integration has to allow two-way data exchange between the CDSS and the EMR, which requires the EMR also to expose a set of interfaces. Further, the CDSS itself needs integration with services on which it depends for its functionality. In the semantic data capture initiative (SDCI) project, we integrated, Proteus (http://proteme.org), an open source, process-oriented clinical decision support system with Henry Ford Health System's EMR, Care Plus. The effort involved addressing some of these challenges and some that are unique to a system like Proteus.
如果与临床决策支持系统(CDSS)相结合，电子病历系统(EMRs)可以对临床医生更具吸引力。然而，在发展信用违约支持服务时，必须考虑到整合，这样，它们不仅可以与本地的电子病历整合，也可以与其他国家开发的电子病历整合。如果CDSS设计得很好，Web服务技术可以改善集成的挑战，但是仍然需要考虑其他几个问题。集成必须允许在CDSS和EMR之间进行双向数据交换，这也要求EMR公开一组接口。此外，CDSS本身需要与其功能所依赖的服务集成。在语义数据捕获计划(SDCI)项目中，我们将Proteus (http://proteme.org)，一个开源的、面向流程的临床决策支持系统与Henry Ford Health system的EMR Care Plus集成在一起。我们的工作涉及解决其中的一些挑战，以及一些像Proteus这样的系统所特有的挑战。
{"title":"Interoperability and Integration Considerations for a Process-Oriented Clinical Decision Support System","authors":"Hemant Shah, Ganesh Krishnan, P. Williams, A. Vogler, R. D. Allard, P. Nadkarni","doi":"10.1109/SERVICES.2011.14","DOIUrl":"https://doi.org/10.1109/SERVICES.2011.14","url":null,"abstract":"Electronic medical record systems (EMRs) can be made more attractive to the clinicians if Clinical Decision Support Systems (CDSS) are integrated with them. However, CDSS have to be developed with integration in mind, such that they may be integrated not just with the local EMR but EMRs developed by others as well. Web Services Technology ameliorates the challenge of integration if the CDSS is well-designed but several other issues still need to be considered. The integration has to allow two-way data exchange between the CDSS and the EMR, which requires the EMR also to expose a set of interfaces. Further, the CDSS itself needs integration with services on which it depends for its functionality. In the semantic data capture initiative (SDCI) project, we integrated, Proteus (http://proteme.org), an open source, process-oriented clinical decision support system with Henry Ford Health System's EMR, Care Plus. The effort involved addressing some of these challenges and some that are unique to a system like Proteus.","PeriodicalId":429726,"journal":{"name":"2011 IEEE World Congress on Services","volume":"55 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125255341","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-07-04DOI: 10.1109/SERVICES.2011.106
R. Maule, W. Lewis
We present a systematic method of measuring performance risk for specific implementations of an Enterprise SOA within a Cloud computing framework. The U.S. Department of Defense (DOD) is determining performance baselines for distributed architectures that are internally organized using service interfaces. Such architectures are essential for highly reliable and life critical Cloud applications. We provide baseline performance for an architecture comprised of service components from the service interfaces provided by several major commercial software vendors. Transition from legacy systems into Service-Oriented Architecture (SOA) systems necessitates an infrastructure of federated components. This paper documents a means to measure such an infrastructure. Baselines provide a foundation for assessment of future services including Quality of Service (QoS) contracts and assessment of fused web services within heterogeneous SOA or Cloud architectures.
{"title":"Performance and QoS in Service-Based Systems","authors":"R. Maule, W. Lewis","doi":"10.1109/SERVICES.2011.106","DOIUrl":"https://doi.org/10.1109/SERVICES.2011.106","url":null,"abstract":"We present a systematic method of measuring performance risk for specific implementations of an Enterprise SOA within a Cloud computing framework. The U.S. Department of Defense (DOD) is determining performance baselines for distributed architectures that are internally organized using service interfaces. Such architectures are essential for highly reliable and life critical Cloud applications. We provide baseline performance for an architecture comprised of service components from the service interfaces provided by several major commercial software vendors. Transition from legacy systems into Service-Oriented Architecture (SOA) systems necessitates an infrastructure of federated components. This paper documents a means to measure such an infrastructure. Baselines provide a foundation for assessment of future services including Quality of Service (QoS) contracts and assessment of fused web services within heterogeneous SOA or Cloud architectures.","PeriodicalId":429726,"journal":{"name":"2011 IEEE World Congress on Services","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114904061","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-07-04DOI: 10.1109/SERVICES.2011.72
F. Lins, Robson W. A. Medeiros, Bruno Silva, Andre R. R. Souza, David Aragão, J. Damasceno, P. Maciel, N. Rosa, Bryan Stephenson, Jun Yu Li
Cloud Computing, Business Process Modeling (BPM) and Service-oriented architectures (SOA) are playing a relevant role in the evolution of Information Technology (IT). A considerable number of system developers are using Cloud technologies to deploy and make available systems over the Internet. Business Process Management standards are being widely used to model business requirements. In addition, SOA-based systems are considered an interesting approach to execute high-level business process specifications. Based on the fact that business processes are executed, usually, using services available in network environments, security requirements should be considered, especially when dealing with sensitive data (e.g., credit card information or personal data). Despite the increasing need for specifying security mechanisms in web service compositions in the Cloud, this topic remains a challenge for many reasons, including the known difficulty of expressing security requirements at a business level and the enforcement of such requirements at an execution level in a cloud environment. This work presents an environment to collaboratively model business processes considering security requirements and to automatically deploy them in the Cloud with security requirements enforcement. The business process is realized through the utilization of web service composition. This environment consists of a set of tools to support the business process modeling and secure service composition execution in the Cloud. Security-related information can be shared among different users in the Cloud and used to enable the activation and configuration of security mechanisms. The proposed approach is showcased in a Virtual Travel Agency scenario to show its feasibility.
{"title":"SSC4Cloud Tooling: An Integrated Environment for the Development of Business Processes with Security Requirements in the Cloud","authors":"F. Lins, Robson W. A. Medeiros, Bruno Silva, Andre R. R. Souza, David Aragão, J. Damasceno, P. Maciel, N. Rosa, Bryan Stephenson, Jun Yu Li","doi":"10.1109/SERVICES.2011.72","DOIUrl":"https://doi.org/10.1109/SERVICES.2011.72","url":null,"abstract":"Cloud Computing, Business Process Modeling (BPM) and Service-oriented architectures (SOA) are playing a relevant role in the evolution of Information Technology (IT). A considerable number of system developers are using Cloud technologies to deploy and make available systems over the Internet. Business Process Management standards are being widely used to model business requirements. In addition, SOA-based systems are considered an interesting approach to execute high-level business process specifications. Based on the fact that business processes are executed, usually, using services available in network environments, security requirements should be considered, especially when dealing with sensitive data (e.g., credit card information or personal data). Despite the increasing need for specifying security mechanisms in web service compositions in the Cloud, this topic remains a challenge for many reasons, including the known difficulty of expressing security requirements at a business level and the enforcement of such requirements at an execution level in a cloud environment. This work presents an environment to collaboratively model business processes considering security requirements and to automatically deploy them in the Cloud with security requirements enforcement. The business process is realized through the utilization of web service composition. This environment consists of a set of tools to support the business process modeling and secure service composition execution in the Cloud. Security-related information can be shared among different users in the Cloud and used to enable the activation and configuration of security mechanisms. The proposed approach is showcased in a Virtual Travel Agency scenario to show its feasibility.","PeriodicalId":429726,"journal":{"name":"2011 IEEE World Congress on Services","volume":"18 5","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114033692","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-07-04DOI: 10.1109/SERVICES.2011.17
Q. Lu
Runtime adaptation of Web service compositions can usually be done in several ways, so it is necessary to decide which adaptation approach to take. The PhD research presented in this paper provides a novel decision making approach, new management algorithms, and a middleware architecture for runtime adaptation of Web service compositions in ways that maximize business value, while satisfying all given constraints. All necessary information about possible adaptations and their business metrics are specified as policies in the WS-Policy4MASC language and the optimization problem is modeled in the powerful constraint programming language MiniZinc. The decision making algorithms integrated into the MiniZnMASC middleware allows it to determine how to adapt each Web service composition instance so the overall business value is maximized, while satisfying all given constraints (e.g., about resource limitations). Experiments with the MiniZnMASC prototype showed that the new solutions are feasible, functionally correct, business beneficial, with low performance overhead, and with linear scalability.
{"title":"Autonomic Business-Driven Decision Making for Adaptation of Web Service Compositions","authors":"Q. Lu","doi":"10.1109/SERVICES.2011.17","DOIUrl":"https://doi.org/10.1109/SERVICES.2011.17","url":null,"abstract":"Runtime adaptation of Web service compositions can usually be done in several ways, so it is necessary to decide which adaptation approach to take. The PhD research presented in this paper provides a novel decision making approach, new management algorithms, and a middleware architecture for runtime adaptation of Web service compositions in ways that maximize business value, while satisfying all given constraints. All necessary information about possible adaptations and their business metrics are specified as policies in the WS-Policy4MASC language and the optimization problem is modeled in the powerful constraint programming language MiniZinc. The decision making algorithms integrated into the MiniZnMASC middleware allows it to determine how to adapt each Web service composition instance so the overall business value is maximized, while satisfying all given constraints (e.g., about resource limitations). Experiments with the MiniZnMASC prototype showed that the new solutions are feasible, functionally correct, business beneficial, with low performance overhead, and with linear scalability.","PeriodicalId":429726,"journal":{"name":"2011 IEEE World Congress on Services","volume":"06 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129724878","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-07-04DOI: 10.1109/SERVICES.2011.64
Congchong Liu, Shujia Zhou
MapReduce, which was introduced by Google, provides two functional interfaces, Map and Reduce, for a user to write the user-specific code to process the large amount of data. It has been widely deployed in cloud computing systems. The parallel tasks, data partition, and data transit are automatically managed by its runtime system. This paper proposes a solution to optimize the MapReduce program model and demonstrate it with X10. We develop an adaptive load distribution scheme to balance the load on each node and consequently reduce across-node communication cost occurring in the Reduce function. In addition, we exploit shared-memory in each node to further reduce the communication cost with multi-core programming.
{"title":"Local and Global Optimization of MapReduce Program Model","authors":"Congchong Liu, Shujia Zhou","doi":"10.1109/SERVICES.2011.64","DOIUrl":"https://doi.org/10.1109/SERVICES.2011.64","url":null,"abstract":"MapReduce, which was introduced by Google, provides two functional interfaces, Map and Reduce, for a user to write the user-specific code to process the large amount of data. It has been widely deployed in cloud computing systems. The parallel tasks, data partition, and data transit are automatically managed by its runtime system. This paper proposes a solution to optimize the MapReduce program model and demonstrate it with X10. We develop an adaptive load distribution scheme to balance the load on each node and consequently reduce across-node communication cost occurring in the Reduce function. In addition, we exploit shared-memory in each node to further reduce the communication cost with multi-core programming.","PeriodicalId":429726,"journal":{"name":"2011 IEEE World Congress on Services","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114308536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-07-04DOI: 10.1109/SERVICES.2011.95
Edmund Kohlwey, Abel Sussman, J. Trost, Amber Maurer
This paper presents a prototype system for generalized searching of cloud-scale biometric data as well as an application of this system to the task of matching a collection of synthetic human iris images. First, we discuss growth trends in the biometric technology market that warrant the application of new data storage and retrieval technologies. Next, we discuss the considerations that motivated our decisions when designing our prototype framework. We then discuss the underlying open source components that we selected for the prototype, its API design, and the relationship between the API and the underlying architecture. We provide details of our implementation of human iris matching within our prototype framework. Finally, we discuss opportunities for future research.
{"title":"Leveraging the Cloud for Big Data Biometrics: Meeting the Performance Requirements of the Next Generation Biometric Systems","authors":"Edmund Kohlwey, Abel Sussman, J. Trost, Amber Maurer","doi":"10.1109/SERVICES.2011.95","DOIUrl":"https://doi.org/10.1109/SERVICES.2011.95","url":null,"abstract":"This paper presents a prototype system for generalized searching of cloud-scale biometric data as well as an application of this system to the task of matching a collection of synthetic human iris images. First, we discuss growth trends in the biometric technology market that warrant the application of new data storage and retrieval technologies. Next, we discuss the considerations that motivated our decisions when designing our prototype framework. We then discuss the underlying open source components that we selected for the prototype, its API design, and the relationship between the API and the underlying architecture. We provide details of our implementation of human iris matching within our prototype framework. Finally, we discuss opportunities for future research.","PeriodicalId":429726,"journal":{"name":"2011 IEEE World Congress on Services","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121768091","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-07-04DOI: 10.1109/SERVICES.2011.53
R. Dantas, Ernani Azevedo, C. Dias, Thiago Lima, D. Sadok, C. Kamienski, B. Ohlman
Service composition is an important feature enabled by Service Oriented Computing but it remains mainly a manual process. Proposals for complete or partial automation of composition exist, the majority relying on artificial intelligence-based planning, automated proof techniques, or graph-based solutions. Although sound, these approaches present practical issues that prevent their use in production scenarios. This paper presents a composition tool that allows for partial service specification along with a composition algorithm based on the semantic matching of services inputs and outputs. Early performance numbers are also presented.
{"title":"Facilitating Service Creation via Partial Specification and Automated Composition","authors":"R. Dantas, Ernani Azevedo, C. Dias, Thiago Lima, D. Sadok, C. Kamienski, B. Ohlman","doi":"10.1109/SERVICES.2011.53","DOIUrl":"https://doi.org/10.1109/SERVICES.2011.53","url":null,"abstract":"Service composition is an important feature enabled by Service Oriented Computing but it remains mainly a manual process. Proposals for complete or partial automation of composition exist, the majority relying on artificial intelligence-based planning, automated proof techniques, or graph-based solutions. Although sound, these approaches present practical issues that prevent their use in production scenarios. This paper presents a composition tool that allows for partial service specification along with a composition algorithm based on the semantic matching of services inputs and outputs. Early performance numbers are also presented.","PeriodicalId":429726,"journal":{"name":"2011 IEEE World Congress on Services","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124227568","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2011-07-04DOI: 10.1109/SERVICES.2011.57
Michael Agun, S. Bowers
Many scientific workflow systems are built on dataflow-based models of computation in which data drives the execution of workflow components. An advantage of using dataflow models is their straightforward semantics (which includes support for branching, merging, and looping) and their ability to concurrently execute workflow steps. However, for many data-intensive workflows the dataflow model often requires data buffering. Current systems largely perform buffering through in-memory queues which can lead to buffer overflow and performance degradation as queues reach capacity (e.g., because of paging). We describe an alternative framework that leverages external storage to implement buffers (which we refer to as persistent queues) within data-intensive scientific workflows. Our framework can easily be used with different underlying storage technologies, and we consider and evaluate three distinct approaches: a traditional relational database implementation, a non-relational implementation designed for fast reads and writes, and a specialized approach that can further reduce external buffering overhead. In addition, the use of persistent queues can provide detailed provenance information ``for free'' by capturing the input and output information of each workflow component during workflow execution. Although many systems provide such provenance information, we show how this information can be captured both efficiently and can be used to improve overall workflow performance through persistent queues.
{"title":"Approaches for Implementing Persistent Queues within Data-Intensive Scientific Workflows","authors":"Michael Agun, S. Bowers","doi":"10.1109/SERVICES.2011.57","DOIUrl":"https://doi.org/10.1109/SERVICES.2011.57","url":null,"abstract":"Many scientific workflow systems are built on dataflow-based models of computation in which data drives the execution of workflow components. An advantage of using dataflow models is their straightforward semantics (which includes support for branching, merging, and looping) and their ability to concurrently execute workflow steps. However, for many data-intensive workflows the dataflow model often requires data buffering. Current systems largely perform buffering through in-memory queues which can lead to buffer overflow and performance degradation as queues reach capacity (e.g., because of paging). We describe an alternative framework that leverages external storage to implement buffers (which we refer to as persistent queues) within data-intensive scientific workflows. Our framework can easily be used with different underlying storage technologies, and we consider and evaluate three distinct approaches: a traditional relational database implementation, a non-relational implementation designed for fast reads and writes, and a specialized approach that can further reduce external buffering overhead. In addition, the use of persistent queues can provide detailed provenance information ``for free'' by capturing the input and output information of each workflow component during workflow execution. Although many systems provide such provenance information, we show how this information can be captured both efficiently and can be used to improve overall workflow performance through persistent queues.","PeriodicalId":429726,"journal":{"name":"2011 IEEE World Congress on Services","volume":"62 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2011-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124384540","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：