首页 > 最新文献

Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.最新文献

英文 中文
Distributed hybrid earthquake engineering experiments: experiences with a ground-shaking grid application 分布式混合地震工程试验:地震动网格应用经验
L. Pearlman, C. Kesselman, S. Gullapalli, B. Spencer, J. Futrelle, K. Ricker, Ian T Foster, P. Hubbard, C. Severance
Earthquake engineers have traditionally investigated the behavior of structures with either computational simulations or physical experiments. Recently, a new hybrid approach has been proposed that allows tests to be decomposed into independent substructures that can be located at different test facilities, tested separately, and integrated via a computational simulation. We describe a grid-based architecture for performing such novel distributed hybrid computational/physical experiments. We discuss the requirements that underlie this extremely challenging application of grid technologies, describe our architecture and implementation, and discuss our experiences with the application of this architecture within an unprecedented earthquake engineering test that coupled large-scale physical experiments in Illinois and Colorado with a computational simulation. Our results point to the remarkable impacts that grid technologies can have on the practice of engineering, and also contribute to our understanding of how to build and deploy effective grid applications.
地震工程师传统上通过计算模拟或物理实验来研究结构的行为。最近,提出了一种新的混合方法,允许将测试分解为独立的子结构,这些子结构可以位于不同的测试设施中,单独测试,并通过计算模拟集成。我们描述了一种基于网格的架构,用于执行这种新颖的分布式混合计算/物理实验。我们讨论了这种极具挑战性的网格技术应用的需求,描述了我们的架构和实现,并讨论了我们在前所未有的地震工程测试中应用该架构的经验,该测试将伊利诺斯州和科罗拉多州的大规模物理实验与计算模拟相结合。我们的研究结果指出了网格技术对工程实践的显著影响,也有助于我们理解如何构建和部署有效的网格应用程序。
{"title":"Distributed hybrid earthquake engineering experiments: experiences with a ground-shaking grid application","authors":"L. Pearlman, C. Kesselman, S. Gullapalli, B. Spencer, J. Futrelle, K. Ricker, Ian T Foster, P. Hubbard, C. Severance","doi":"10.1109/HPDC.2004.11","DOIUrl":"https://doi.org/10.1109/HPDC.2004.11","url":null,"abstract":"Earthquake engineers have traditionally investigated the behavior of structures with either computational simulations or physical experiments. Recently, a new hybrid approach has been proposed that allows tests to be decomposed into independent substructures that can be located at different test facilities, tested separately, and integrated via a computational simulation. We describe a grid-based architecture for performing such novel distributed hybrid computational/physical experiments. We discuss the requirements that underlie this extremely challenging application of grid technologies, describe our architecture and implementation, and discuss our experiences with the application of this architecture within an unprecedented earthquake engineering test that coupled large-scale physical experiments in Illinois and Colorado with a computational simulation. Our results point to the remarkable impacts that grid technologies can have on the practice of engineering, and also contribute to our understanding of how to build and deploy effective grid applications.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"112 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117137413","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 60
Differential serialization for optimized SOAP performance 用于优化SOAP性能的差分序列化
N. Abu-Ghazaleh, M. Lewis, M. Govindaraju
The SOAP protocol has emerged as a Web Service communication standard, providing simplicity, robustness, and extensibility. SOAP's relatively poor performance threatens to limit its usefulness, especially for high-performance scientific applications. The serialization of outgoing messages, which includes conversion of in-memory data types to XML-based string format and the packing of this data into message buffers, is a primary SOAP performance bottleneck. We describe the design and implementation of differential serialization, a SOAP optimization technique that can help bypass the serialization step for messages similar to those previously sent by a SOAP client or previously returned by a SOAP-based Web Service. The approach requires no changes to the SOAP protocol. Our implementation and performance study demonstrate the technique *s potential, showing a substantial performance improvement over widely used SOAP toolkits that do not employ the optimization. We identify several factors that determine the usefulness and applicability of differential serialization, present a set of techniques for increasing the situations in which it can be used, and explore the design space of the approach.
SOAP协议是作为Web服务通信标准出现的,它提供了简单性、健壮性和可扩展性。SOAP相对较差的性能可能会限制它的用途,特别是对于高性能的科学应用程序。传出消息的序列化(包括将内存中的数据类型转换为基于xml的字符串格式以及将这些数据打包到消息缓冲区中)是SOAP性能的主要瓶颈。我们描述了差分序列化的设计和实现,差分序列化是一种SOAP优化技术,它可以帮助绕过消息的序列化步骤,这些消息类似于以前由SOAP客户机发送的消息或以前由基于SOAP的Web服务返回的消息。该方法不需要更改SOAP协议。我们的实现和性能研究证明了该技术的潜力,显示了相对于未采用优化的广泛使用的SOAP工具包的实质性性能改进。我们确定了决定差分序列化的有用性和适用性的几个因素,提出了一组技术来增加可以使用它的情况,并探索了该方法的设计空间。
{"title":"Differential serialization for optimized SOAP performance","authors":"N. Abu-Ghazaleh, M. Lewis, M. Govindaraju","doi":"10.1109/HPDC.2004.8","DOIUrl":"https://doi.org/10.1109/HPDC.2004.8","url":null,"abstract":"The SOAP protocol has emerged as a Web Service communication standard, providing simplicity, robustness, and extensibility. SOAP's relatively poor performance threatens to limit its usefulness, especially for high-performance scientific applications. The serialization of outgoing messages, which includes conversion of in-memory data types to XML-based string format and the packing of this data into message buffers, is a primary SOAP performance bottleneck. We describe the design and implementation of differential serialization, a SOAP optimization technique that can help bypass the serialization step for messages similar to those previously sent by a SOAP client or previously returned by a SOAP-based Web Service. The approach requires no changes to the SOAP protocol. Our implementation and performance study demonstrate the technique *s potential, showing a substantial performance improvement over widely used SOAP toolkits that do not employ the optimization. We identify several factors that determine the usefulness and applicability of differential serialization, present a set of techniques for increasing the situations in which it can be used, and explore the design space of the approach.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"17 8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125791470","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 121
Building self-configuring services using service-specific knowledge 使用特定于服务的知识构建自配置服务
An-Cheng Huang, P. Steenkiste
A self-configuring sendee can automatically leverage distributed sendee components and resources to compose an optimal configuration according to both the requirements of a particular user and the system characteristics. One major challenge for building such sendees is how to bring in sendee-specific knowledge, e.g., what components are needed and optimization criteria to use, while still allowing reuse of common sendee composition functionalities. We present an architecture in which service developers express their service-specific knowledge in the form of a service recipe that is used by a generic synthesizer to perform sendee composition automatically. We apply our approach to three different services to illustrate the flexibility and simplicity of the recipe representation. We use simulations based on Internet measurements to evaluate how an appropriate optimization algorithm can be selected according to a developer's sendee-specific trade-off between optimality and cost of optimization.
自配置发送者可以自动利用分布式发送者组件和资源,根据特定用户的需求和系统特征组合最佳配置。构建这样的发送程序的一个主要挑战是如何引入特定于发送程序的知识,例如,需要哪些组件和要使用的优化标准,同时仍然允许重用常见的发送程序组合功能。我们提出了一个体系结构,在这个体系结构中,服务开发人员以服务配方的形式表达他们特定于服务的知识,通用合成器使用该服务配方来自动执行发送者组合。我们将这种方法应用于三个不同的服务,以说明配方表示的灵活性和简单性。我们使用基于互联网测量的模拟来评估如何根据开发人员在优化性和优化成本之间的特定send -specific权衡选择合适的优化算法。
{"title":"Building self-configuring services using service-specific knowledge","authors":"An-Cheng Huang, P. Steenkiste","doi":"10.1109/HPDC.2004.6","DOIUrl":"https://doi.org/10.1109/HPDC.2004.6","url":null,"abstract":"A self-configuring sendee can automatically leverage distributed sendee components and resources to compose an optimal configuration according to both the requirements of a particular user and the system characteristics. One major challenge for building such sendees is how to bring in sendee-specific knowledge, e.g., what components are needed and optimization criteria to use, while still allowing reuse of common sendee composition functionalities. We present an architecture in which service developers express their service-specific knowledge in the form of a service recipe that is used by a generic synthesizer to perform sendee composition automatically. We apply our approach to three different services to illustrate the flexibility and simplicity of the recipe representation. We use simulations based on Internet measurements to evaluate how an appropriate optimization algorithm can be selected according to a developer's sendee-specific trade-off between optimality and cost of optimization.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132380848","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 34
Automated cluster-based Web service performance tuning 基于集群的自动Web服务性能调优
I. Chung, J. Hollingsworth
Active harmony provides a way to automate performance tuning. We apply the Active Harmony system to improve the performance of a cluster-based web service system. The performance improvement cannot easily be achieved by tuning individual components for such a system. The experimental results show that there is no single configuration for the system that performs well for all kinds of workloads. By tuning the parameters, Active Harmony helps the system adapt to different workloads and improve the performance up to 16%. For scalability, we demonstrate how to reduce the time when tuning a large system with many tunable parameters. Finally an algorithm is proposed to automatically adjust the structure of cluster-based web systems, and the system throughput is improved up to 70% using this technique.
主动协调提供了一种自动化性能调优的方法。我们应用Active Harmony系统来提高一个基于集群的web服务系统的性能。通过调优这样一个系统的单个组件,不容易实现性能改进。实验结果表明,没有一种单一的系统配置可以很好地适应各种工作负载。通过调整参数,Active Harmony可以帮助系统适应不同的工作负载,并将性能提高16%。对于可伸缩性,我们将演示如何在调优具有许多可调参数的大型系统时减少时间。最后提出了一种基于集群的web系统结构自动调整算法,该算法使系统吞吐量提高了70%。
{"title":"Automated cluster-based Web service performance tuning","authors":"I. Chung, J. Hollingsworth","doi":"10.1109/HPDC.2004.4","DOIUrl":"https://doi.org/10.1109/HPDC.2004.4","url":null,"abstract":"Active harmony provides a way to automate performance tuning. We apply the Active Harmony system to improve the performance of a cluster-based web service system. The performance improvement cannot easily be achieved by tuning individual components for such a system. The experimental results show that there is no single configuration for the system that performs well for all kinds of workloads. By tuning the parameters, Active Harmony helps the system adapt to different workloads and improve the performance up to 16%. For scalability, we demonstrate how to reduce the time when tuning a large system with many tunable parameters. Finally an algorithm is proposed to automatically adjust the structure of cluster-based web systems, and the system throughput is improved up to 70% using this technique.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"254 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117324222","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 49
Performance evaluation of rate-based join window sizing for asynchronous data streams 异步数据流基于速率的连接窗口大小的性能评估
N. Vijayakumar, Beth Plale
Our work is motivated by the large number of data stream sources that define mesoscale meteorology where asynchronous streams are commonplace. Techniques for performing filtering, aggregation, and transformation on multiple streams must be effective for the case of asynchronous streams. Rate Sizing algorithm (RS-Algo) links the number of events waiting to participate in a join to the rate of the streams responsible for their delivery. In this poster, we show the results of performance evaluation of the RS-Algo. The gains in memory utilization are largest under asynchronous streams.
我们的工作是由大量的数据流源驱动的,这些数据流源定义了中尺度气象学,其中异步流是常见的。在多个流上执行过滤、聚合和转换的技术必须对异步流有效。速率分级算法(RS-Algo)将等待参与join的事件数量与负责它们交付的流的速率联系起来。在这张海报中,我们展示了RS-Algo的性能评估结果。在异步流下,内存利用率的提高最大。
{"title":"Performance evaluation of rate-based join window sizing for asynchronous data streams","authors":"N. Vijayakumar, Beth Plale","doi":"10.1109/HPDC.2004.28","DOIUrl":"https://doi.org/10.1109/HPDC.2004.28","url":null,"abstract":"Our work is motivated by the large number of data stream sources that define mesoscale meteorology where asynchronous streams are commonplace. Techniques for performing filtering, aggregation, and transformation on multiple streams must be effective for the case of asynchronous streams. Rate Sizing algorithm (RS-Algo) links the number of events waiting to participate in a join to the rate of the streams responsible for their delivery. In this poster, we show the results of performance evaluation of the RS-Algo. The gains in memory utilization are largest under asynchronous streams.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"79 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124578102","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Balancing risk and reward in a market-based task service 在基于市场的任务服务中平衡风险和回报
David E. Irwin, Laura E. Grit, J. Chase
We investigate the question of scheduling tasks according to a user-centric value metric-called yield or utility. User value is an attractive basis for allocating shared computing resources, and is fundamental to economic approaches to resource management in linked clusters or grids. Even so, commonly used batch schedulers do not yet support value-based scheduling, and there has been little study of its use in a market-based grid setting. In part this is because scheduling to maximize time-varying value is a difficult problem where even simple formulations are intractable. We present improved heuristics for value-based task scheduling using a simple but rich formulation of value, in which a task's yield decays linearly with its waiting time. We also show the role of value-based scheduling heuristics in a framework for market-based bidding and admission control, in which clients negotiate for task services from multiple grid sites. Our approach follows an investment metaphor: the heuristics balance the risk of future costs against the potential for gains in accepting and scheduling tasks. In particular, we show the importance of opportunity cost, and the impact of risk due to uncertainty in the future job mix.
我们根据以用户为中心的价值度量(称为产出或效用)来研究任务调度问题。用户价值是分配共享计算资源的一个有吸引力的基础,也是在关联集群或网格中进行资源管理的经济方法的基础。即便如此,常用的批调度程序还不支持基于值的调度,并且很少有关于它在基于市场的网格设置中的使用的研究。在某种程度上,这是因为最大化时变值的调度是一个难题,即使是简单的公式也难以处理。我们提出了一种改进的启发式的基于价值的任务调度方法,使用一个简单而丰富的价值公式,其中任务的产量随其等待时间线性衰减。我们还展示了基于价值的调度启发式在基于市场的投标和准入控制框架中的作用,其中客户端从多个网格站点协商任务服务。我们的方法遵循一个投资比喻:启发式方法在接受和调度任务时平衡未来成本的风险和潜在的收益。特别是,我们展示了机会成本的重要性,以及由于未来工作组合的不确定性而带来的风险的影响。
{"title":"Balancing risk and reward in a market-based task service","authors":"David E. Irwin, Laura E. Grit, J. Chase","doi":"10.1109/HPDC.2004.5","DOIUrl":"https://doi.org/10.1109/HPDC.2004.5","url":null,"abstract":"We investigate the question of scheduling tasks according to a user-centric value metric-called yield or utility. User value is an attractive basis for allocating shared computing resources, and is fundamental to economic approaches to resource management in linked clusters or grids. Even so, commonly used batch schedulers do not yet support value-based scheduling, and there has been little study of its use in a market-based grid setting. In part this is because scheduling to maximize time-varying value is a difficult problem where even simple formulations are intractable. We present improved heuristics for value-based task scheduling using a simple but rich formulation of value, in which a task's yield decays linearly with its waiting time. We also show the role of value-based scheduling heuristics in a framework for market-based bidding and admission control, in which clients negotiate for task services from multiple grid sites. Our approach follows an investment metaphor: the heuristics balance the risk of future costs against the potential for gains in accepting and scheduling tasks. In particular, we show the importance of opportunity cost, and the impact of risk due to uncertainty in the future job mix.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"86 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123946800","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 233
The Grid2003 production grid: principles and practice Grid2003生产网格:原理与实践
Ian T Foster, J. Gieraltowski, Scott Gose, N. Maltsev, E. May, Alex Rodriguez, Dinanath Sulakhe, A. Vaniachine, J. Shank, S. Youssef, D. Adams, R. Baker, W. Deng, J. Smith, Dantong Yu, I. Legrand, Suresh Singh, C. Steenberg, Yang Xia, M. Afaq, E. Berman, J. Annis, L. Bauerdick, M. Ernst, I. Fisk, L. Giacchetti, G. Graham, A. Heavey, J. Kaiser, N. Kuropatkin, R. Pordes, V. Sekhri, J. Weigand, Yujun Wu, Keith Baker, Lawrence Sorrillo, J. Huth, Matthew Allen, L. Grundhoefer, J. Hicks, F. Luehring, S. Peck, R. Quick, Stephen C. Simms, G. Fekete, Jan vandenBerg, Kihyeon Cho, Kihwan Kwon, Dongchul Son, Hyoungwoo Park, S. Canon, K. Jackson, D. Konerding, Jason R. Lee, D. Olson, I. Sakrejda, B. Tierney, Mark L. Green, Russ Miller, J. Letts, T. Martin, David Bury, C. Dumitrescu, D. Engh, R. Gardner, M. Mambelli, Y. Smirnov, Jens-S. Vöckler, M. Wilde, Yong Zhao, Xin Zhao, P. Avery, R. Cavanaugh, Bockjoo Kim, C. Prescott, J. Rodriguez, A. Zahn, S. McKee, C. Jordan, James E. Prewett, T. Thomas, H. Severini, Ben Cliff
The Grid2003 Project has deployed a multivirtual organization, application-driven grid laboratory ("Grid3") that has sustained for several months the production-level services required by physics experiments of the Large Hadron Collider at CERN (ATLAS and CMS), the Sloan Digital Sky Survey project, the gravitational wave search experiment LIGO, the BTeV experiment at Fermilab, as well as applications in molecular structure analysis and genome analysis, and computer science research projects in such areas as job and data scheduling. The deployed infrastructure has been operating since November 2003 with 27 sites, a peak of 2800 processors, work loads from 10 different applications exceeding 1300 simultaneous jobs, and data transfers among sites of greater than 2 TB/day. We describe the principles that have guided the development of this unique infrastructure and the practical experiences that have resulted from its creation and use. We discuss application requirements for grid services deployment and configuration, monitoring infrastructure, application performance, metrics, and operational experiences. We also summarize lessons learned.
Grid2003项目已经部署了一个多虚拟组织、应用驱动的网格实验室(“Grid3”),该实验室已经持续了几个月的生产级服务,用于欧洲核子研究中心的大型强子对撞机(ATLAS和CMS)、斯隆数字巡天项目、引力波搜索实验LIGO、费米实验室的BTeV实验,以及分子结构分析和基因组分析方面的应用。以及诸如作业和数据调度等领域的计算机科学研究项目。部署的基础设施自2003年11月以来一直运行,有27个站点,峰值有2800个处理器,来自10个不同应用程序的工作负载超过1300个同时工作,站点之间的数据传输超过2tb /天。我们描述了指导这一独特基础设施发展的原则,以及从其创建和使用中产生的实践经验。我们将讨论网格服务部署和配置、监视基础设施、应用程序性能、度量和操作经验的应用程序需求。我们还总结了经验教训。
{"title":"The Grid2003 production grid: principles and practice","authors":"Ian T Foster, J. Gieraltowski, Scott Gose, N. Maltsev, E. May, Alex Rodriguez, Dinanath Sulakhe, A. Vaniachine, J. Shank, S. Youssef, D. Adams, R. Baker, W. Deng, J. Smith, Dantong Yu, I. Legrand, Suresh Singh, C. Steenberg, Yang Xia, M. Afaq, E. Berman, J. Annis, L. Bauerdick, M. Ernst, I. Fisk, L. Giacchetti, G. Graham, A. Heavey, J. Kaiser, N. Kuropatkin, R. Pordes, V. Sekhri, J. Weigand, Yujun Wu, Keith Baker, Lawrence Sorrillo, J. Huth, Matthew Allen, L. Grundhoefer, J. Hicks, F. Luehring, S. Peck, R. Quick, Stephen C. Simms, G. Fekete, Jan vandenBerg, Kihyeon Cho, Kihwan Kwon, Dongchul Son, Hyoungwoo Park, S. Canon, K. Jackson, D. Konerding, Jason R. Lee, D. Olson, I. Sakrejda, B. Tierney, Mark L. Green, Russ Miller, J. Letts, T. Martin, David Bury, C. Dumitrescu, D. Engh, R. Gardner, M. Mambelli, Y. Smirnov, Jens-S. Vöckler, M. Wilde, Yong Zhao, Xin Zhao, P. Avery, R. Cavanaugh, Bockjoo Kim, C. Prescott, J. Rodriguez, A. Zahn, S. McKee, C. Jordan, James E. Prewett, T. Thomas, H. Severini, Ben Cliff","doi":"10.1109/HPDC.2004.36","DOIUrl":"https://doi.org/10.1109/HPDC.2004.36","url":null,"abstract":"The Grid2003 Project has deployed a multivirtual organization, application-driven grid laboratory (\"Grid3\") that has sustained for several months the production-level services required by physics experiments of the Large Hadron Collider at CERN (ATLAS and CMS), the Sloan Digital Sky Survey project, the gravitational wave search experiment LIGO, the BTeV experiment at Fermilab, as well as applications in molecular structure analysis and genome analysis, and computer science research projects in such areas as job and data scheduling. The deployed infrastructure has been operating since November 2003 with 27 sites, a peak of 2800 processors, work loads from 10 different applications exceeding 1300 simultaneous jobs, and data transfers among sites of greater than 2 TB/day. We describe the principles that have guided the development of this unique infrastructure and the practical experiences that have resulted from its creation and use. We discuss application requirements for grid services deployment and configuration, monitoring infrastructure, application performance, metrics, and operational experiences. We also summarize lessons learned.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130876664","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 152
Optimal resource-aware deployment planning for component-based distributed applications 为基于组件的分布式应用程序提供资源感知的最佳部署规划
T. Kichkaylo, V. Karamcheti
Component-based approaches are becoming increasingly popular in the areas of adaptive distributed systems, Web services, and grid computing. In each case, the underlying infrastructure needs to address a deployment problem involving the placement of application components onto computational, data, and network resources across a wide-area environment subject to a variety of qualitative and quantitative constraints. In general, the deployment needs to also introduce auxiliary components (e.g., to compress/decompress data, or invoke GridFTP sessions to make data available at a remote site), and reuse preexisting components and data. To provide the flexibility required in the latter case, recently proposed systems such as Sekitei and Pegasus have proposed solutions that rely upon Al planning-based techniques. Although promising, the inherent complexity of Al planning and the fact that constraints governing component deployment often involve nonlinear and nonreversible functions have prevented such solutions from generating deployments in resource-constrained situations and achieving optimality in terms of overall resource usage or other cost metrics. We address both of these shortcomings in the context of the Sekitei system. Our extension relies upon information supplied by a domain expert, which classifies component behavior into a discrete set of levels. This discretization, often justified in practice, permits the planner to identify cost-optimal plans (whose quality improves with the level definitions) without restricting the form of the constraint functions. We describe the modified Sekitei algorithm, and characterize, using a media stream delivery application, its scaling behavior when generating optimal deployments for various network configurations.
基于组件的方法在自适应分布式系统、Web服务和网格计算领域变得越来越流行。在每种情况下,底层基础设施都需要解决一个部署问题,涉及将应用程序组件放置到受各种定性和定量约束的广域环境中的计算、数据和网络资源上。通常,部署还需要引入辅助组件(例如,压缩/解压缩数据,或调用GridFTP会话以使数据在远程站点可用),并重用先前存在的组件和数据。为了提供后一种情况所需的灵活性,最近提出的系统,如Sekitei和Pegasus,已经提出了依赖于基于人工智能规划技术的解决方案。尽管前景很好,但是人工智能计划的固有复杂性以及控制组件部署的约束通常涉及非线性和不可逆函数的事实,阻止了这种解决方案在资源受限的情况下生成部署,并在总体资源使用或其他成本度量方面实现最优。我们在Sekitei系统的背景下解决了这两个缺点。我们的扩展依赖于领域专家提供的信息,该信息将组件行为分类为一组离散的级别。这种离散化在实践中经常被证明是合理的,它允许计划者在不限制约束函数形式的情况下确定成本最优计划(其质量随着级别定义而提高)。我们描述了改进的Sekitei算法,并使用媒体流交付应用程序描述了它在为各种网络配置生成最佳部署时的扩展行为。
{"title":"Optimal resource-aware deployment planning for component-based distributed applications","authors":"T. Kichkaylo, V. Karamcheti","doi":"10.1109/HPDC.2004.25","DOIUrl":"https://doi.org/10.1109/HPDC.2004.25","url":null,"abstract":"Component-based approaches are becoming increasingly popular in the areas of adaptive distributed systems, Web services, and grid computing. In each case, the underlying infrastructure needs to address a deployment problem involving the placement of application components onto computational, data, and network resources across a wide-area environment subject to a variety of qualitative and quantitative constraints. In general, the deployment needs to also introduce auxiliary components (e.g., to compress/decompress data, or invoke GridFTP sessions to make data available at a remote site), and reuse preexisting components and data. To provide the flexibility required in the latter case, recently proposed systems such as Sekitei and Pegasus have proposed solutions that rely upon Al planning-based techniques. Although promising, the inherent complexity of Al planning and the fact that constraints governing component deployment often involve nonlinear and nonreversible functions have prevented such solutions from generating deployments in resource-constrained situations and achieving optimality in terms of overall resource usage or other cost metrics. We address both of these shortcomings in the context of the Sekitei system. Our extension relies upon information supplied by a domain expert, which classifies component behavior into a discrete set of levels. This discretization, often justified in practice, permits the planner to identify cost-optimal plans (whose quality improves with the level definitions) without restricting the form of the constraint functions. We describe the modified Sekitei algorithm, and characterize, using a media stream delivery application, its scaling behavior when generating optimal deployments for various network configurations.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133789071","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 63
Orchestrating and coordinating scientific/engineering workflows using GridShell 使用GridShell编排和协调科学/工程工作流程
Edward Walker, T. Minyard
GridShell maps grid and distributed computing concepts to the UNIX shell programming environment. This allows scientific and engineering users, already familiar with Tenex C Shell (tcsh) and Bourne Again Shell (bash), to quickly use the services provided by these distributed resources. Our demo will show how the scripting capabilities of GridShell can be used to orchestrate and coordinate the workflow of an engineering simulation using an established 2D flow solver.
GridShell将网格和分布式计算概念映射到UNIX shell编程环境。这使得已经熟悉Tenex C Shell (tcsh)和Bourne Again Shell (bash)的科学和工程用户可以快速使用这些分布式资源提供的服务。我们的演示将展示GridShell的脚本功能如何使用已建立的2D流求解器来编排和协调工程仿真的工作流程。
{"title":"Orchestrating and coordinating scientific/engineering workflows using GridShell","authors":"Edward Walker, T. Minyard","doi":"10.1109/HPDC.2004.26","DOIUrl":"https://doi.org/10.1109/HPDC.2004.26","url":null,"abstract":"GridShell maps grid and distributed computing concepts to the UNIX shell programming environment. This allows scientific and engineering users, already familiar with Tenex C Shell (tcsh) and Bourne Again Shell (bash), to quickly use the services provided by these distributed resources. Our demo will show how the scripting capabilities of GridShell can be used to orchestrate and coordinate the workflow of an engineering simulation using an established 2D flow solver.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128160521","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
MPI Ruby with remote memory access 具有远程内存访问的MPI Ruby
Christopher C. Aycock
Advances in communication for parallel programming have yielded one-sided messaging systems. The MPI bindings for Ruby have been augmented to include the remote memory access functions of MPI-2.
并行编程通信的进步产生了单侧消息传递系统。Ruby的MPI绑定已经增强,包括MPI-2的远程内存访问功能。
{"title":"MPI Ruby with remote memory access","authors":"Christopher C. Aycock","doi":"10.1109/HPDC.2004.24","DOIUrl":"https://doi.org/10.1109/HPDC.2004.24","url":null,"abstract":"Advances in communication for parallel programming have yielded one-sided messaging systems. The MPI bindings for Ruby have been augmented to include the remote memory access functions of MPI-2.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128390787","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
期刊
Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1