首页 > 最新文献

Proceedings. IEEE International Conference on Cluster Computing最新文献

英文 中文
BioOpera: cluster-aware computing BioOpera:集群感知计算
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137734
Win Bausch, C. Pautasso, R. Schaeppi, G. Alonso
In this paper we present BioOpera, an extensible process support system for cluster-aware computing. It features an intuitive way to specify computations, as well as improved support for running them over a cluster providing monitoring, persistence, fault tolerance and interaction capabilities without sacrificing efficiency and scalability.
在本文中,我们提出了BioOpera,一个可扩展的进程支持系统,用于集群感知计算。它提供了一种直观的方式来指定计算,并改进了对在集群上运行计算的支持,在不牺牲效率和可伸缩性的情况下提供监视、持久性、容错和交互功能。
{"title":"BioOpera: cluster-aware computing","authors":"Win Bausch, C. Pautasso, R. Schaeppi, G. Alonso","doi":"10.1109/CLUSTR.2002.1137734","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137734","url":null,"abstract":"In this paper we present BioOpera, an extensible process support system for cluster-aware computing. It features an intuitive way to specify computations, as well as improved support for running them over a cluster providing monitoring, persistence, fault tolerance and interaction capabilities without sacrificing efficiency and scalability.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77635797","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 30
Indexing the web - a challenge for supercomputers 索引网络——对超级计算机的挑战
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137763
M. Henzinger
Since January 2002, the Google search engine has been powering an average of 150 million web searches a day, with a peark of over 2000 searches per second. These searches are performed over an index of over 2 billion documents, over 300 million images, and over 700 million Usenet messages. To guarantee fast user response time, Google performs these searches on a cluster of over 10,000 PCs. The main challenages with this architecture are fault-tolerance and the quality of search results. Replication solves the former and the PageRank score is used to advance the latter. The PageRank score is based on an eigenvalue computation of a large matrix that is derived from the web graph and is one of the main contributor to very high quality search results. As Internet use continues to grow, so does the use of the Google search engine. The Google architecture is designed to scale to accommodate the growth in useage as well as the growth of the web.
自2002年1月以来,谷歌搜索引擎每天平均进行1.5亿次网络搜索,每秒搜索次数超过2000次。这些搜索是在超过20亿个文档、超过3亿个图像和超过7亿个Usenet消息的索引上执行的。为了保证快速的用户响应时间,Google在超过10,000台pc的集群上执行这些搜索。这种架构的主要挑战是容错性和搜索结果的质量。复制解决了前者,而PageRank分数用于推进后者。PageRank分数是基于一个大矩阵的特征值计算,该矩阵来源于网络图,是非常高质量搜索结果的主要贡献者之一。随着互联网的使用不断增长,谷歌搜索引擎的使用也在不断增长。Google架构的设计是为了适应用户的增长和网络的增长。
{"title":"Indexing the web - a challenge for supercomputers","authors":"M. Henzinger","doi":"10.1109/CLUSTR.2002.1137763","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137763","url":null,"abstract":"Since January 2002, the Google search engine has been powering an average of 150 million web searches a day, with a peark of over 2000 searches per second. These searches are performed over an index of over 2 billion documents, over 300 million images, and over 700 million Usenet messages. To guarantee fast user response time, Google performs these searches on a cluster of over 10,000 PCs. The main challenages with this architecture are fault-tolerance and the quality of search results. Replication solves the former and the PageRank score is used to advance the latter. The PageRank score is based on an eigenvalue computation of a large matrix that is derived from the web graph and is one of the main contributor to very high quality search results. As Internet use continues to grow, so does the use of the Google search engine. The Google architecture is designed to scale to accommodate the growth in useage as well as the growth of the web.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74746748","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Clusters as large-scale development facilities 作为大型开发设施的集群
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137729
R. Evard, N. Desai, J. Navarro, Daniel Nurmi
In this paper, we describe the use of a cluster as a generalized facility for development. A development facility is a system used primarily for testing and development activities while being operated reliably for many users. We are in the midst of a project to build and operate a large-scale development facility. We discuss our motivation for using clusters in this way and compare the model with a classic computing facility. We describe our experiences and findings from the first phase of this project. Many of these observations are relevant to the design of standard clusters and to future development facilities.
在本文中,我们描述了集群作为一种通用的开发工具的使用。开发设施是主要用于测试和开发活动的系统,同时可为许多用户可靠地操作。我们正在进行一个建设和运营大型开发设施的项目。我们讨论了以这种方式使用集群的动机,并将该模型与经典计算工具进行了比较。我们描述了我们在这个项目第一阶段的经验和发现。这些观察结果中的许多都与标准集群的设计和未来的开发设施有关。
{"title":"Clusters as large-scale development facilities","authors":"R. Evard, N. Desai, J. Navarro, Daniel Nurmi","doi":"10.1109/CLUSTR.2002.1137729","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137729","url":null,"abstract":"In this paper, we describe the use of a cluster as a generalized facility for development. A development facility is a system used primarily for testing and development activities while being operated reliably for many users. We are in the midst of a project to build and operate a large-scale development facility. We discuss our motivation for using clusters in this way and compare the model with a classic computing facility. We describe our experiences and findings from the first phase of this project. Many of these observations are relevant to the design of standard clusters and to future development facilities.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91214538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
Design and implementation of CC-NUMA card II for SCI-based PC clustering 基于sci的PC机集群CC-NUMA卡II的设计与实现
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137739
Soo-Cheol Oh, Sang-Hwa Chung, Hankook Jang
It is extremely important to minimize network access time when constructing a high-performance PC cluster system. For an SCI-based PC cluster it is possible to reduce the network access time by maintaining network cache in each cluster node. This paper presents the second version CC-NUMA card (CC-NUMA card II) that utilizes network cache for SCI-based PC clustering. The CC-NUMA card II is directly plugged into the PCI slot of each node, and contains shared memory, network cache, a shared memory control module and network control module. The network cache is maintained for shared memory on the PCI bus of cluster nodes. The coherency mechanism between network cache and shared memory is based on the IEEE SCI standard. In previous research, the first version CC-NUMA card (CC-NUMA card I) was developed. The CC-NUMA card I adopting Dolphin's PCI-SCI card as the network control module caused overhead in exchanging data between the remote nodes. In this paper, the overhead is removed by developing the CC-NUMA card II that combines the shared memory control module and network control module in a single card. Throughout the experiment with the SPLASH-2 benchmark suite, the CC-NUMA card II based PC cluster shows better performance than a NUMA system based on Dolphin's PCI-SCI card.
在构建高性能PC集群系统时,最小化网络访问时间是非常重要的。对于基于sci的PC集群,可以通过在每个集群节点中维护网络缓存来减少网络访问时间。本文提出了第二版CC-NUMA卡(CC-NUMA卡II),它利用网络缓存实现基于sci的PC机集群。CC-NUMA卡II直接插入每个节点的PCI插槽,包含共享内存、网络缓存、共享内存控制模块和网络控制模块。为集群节点的PCI总线上的共享内存维护网络缓存。网络缓存与共享内存之间的一致性机制基于IEEE SCI标准。在以往的研究中,开发了第一个版本的CC-NUMA卡(CC-NUMA卡I)。采用Dolphin的PCI-SCI卡作为网络控制模块的CC-NUMA卡I造成了远程节点间数据交换的开销。本文通过开发将共享内存控制模块和网络控制模块结合在一块卡中的CC-NUMA卡II来消除开销。在splash2基准测试套件的整个实验中,基于CC-NUMA卡II的PC集群表现出比基于Dolphin的PCI-SCI卡的NUMA系统更好的性能。
{"title":"Design and implementation of CC-NUMA card II for SCI-based PC clustering","authors":"Soo-Cheol Oh, Sang-Hwa Chung, Hankook Jang","doi":"10.1109/CLUSTR.2002.1137739","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137739","url":null,"abstract":"It is extremely important to minimize network access time when constructing a high-performance PC cluster system. For an SCI-based PC cluster it is possible to reduce the network access time by maintaining network cache in each cluster node. This paper presents the second version CC-NUMA card (CC-NUMA card II) that utilizes network cache for SCI-based PC clustering. The CC-NUMA card II is directly plugged into the PCI slot of each node, and contains shared memory, network cache, a shared memory control module and network control module. The network cache is maintained for shared memory on the PCI bus of cluster nodes. The coherency mechanism between network cache and shared memory is based on the IEEE SCI standard. In previous research, the first version CC-NUMA card (CC-NUMA card I) was developed. The CC-NUMA card I adopting Dolphin's PCI-SCI card as the network control module caused overhead in exchanging data between the remote nodes. In this paper, the overhead is removed by developing the CC-NUMA card II that combines the shared memory control module and network control module in a single card. Throughout the experiment with the SPLASH-2 benchmark suite, the CC-NUMA card II based PC cluster shows better performance than a NUMA system based on Dolphin's PCI-SCI card.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84551825","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Design of a middleware-based cluster management platform with task management and migration 基于中间件的集群管理平台,具有任务管理和迁移功能
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137788
F. Turck, S. Vanhastel, P. Thysebaert, B. Volckaert, P. Demeester, B. Dhoedt
In this paper, we address the design and implementation of a generic and scalable platform for efficient management of computational resources. The developed platform is called the Intelligent Agent Platform. Its architecture is based on middleware technology in order to ensure easy distribution of the software components between the participating workstations and to exploit advanced software techniques. The computational tasks are referred to as agents, defined as software components that are capable of executing particular algorithms on input data. The platform offers advanced features such as transparent task management, load balancing, run time compilation of agent code and task migration and is therefore denoted by the adjective "Intelligent". The architecture of the platform will be outlined from a computational point of view and each component will be described in detail. Furthermore, some important design issues of the platform are covered and a performance evaluation is presented.
在本文中,我们讨论了一个通用的、可扩展的平台的设计和实现,用于有效地管理计算资源。开发的平台称为智能代理平台。它的体系结构基于中间件技术,以确保软件组件在参与的工作站之间易于分发,并利用先进的软件技术。计算任务被称为代理,定义为能够对输入数据执行特定算法的软件组件。该平台提供了一些高级特性,如透明的任务管理、负载平衡、代理代码的运行时编译和任务迁移,因此被称为“智能”。将从计算的角度概述平台的体系结构,并详细描述每个组件。此外,还讨论了该平台的一些重要设计问题,并给出了性能评价。
{"title":"Design of a middleware-based cluster management platform with task management and migration","authors":"F. Turck, S. Vanhastel, P. Thysebaert, B. Volckaert, P. Demeester, B. Dhoedt","doi":"10.1109/CLUSTR.2002.1137788","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137788","url":null,"abstract":"In this paper, we address the design and implementation of a generic and scalable platform for efficient management of computational resources. The developed platform is called the Intelligent Agent Platform. Its architecture is based on middleware technology in order to ensure easy distribution of the software components between the participating workstations and to exploit advanced software techniques. The computational tasks are referred to as agents, defined as software components that are capable of executing particular algorithms on input data. The platform offers advanced features such as transparent task management, load balancing, run time compilation of agent code and task migration and is therefore denoted by the adjective \"Intelligent\". The architecture of the platform will be outlined from a computational point of view and each component will be described in detail. Furthermore, some important design issues of the platform are covered and a performance evaluation is presented.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83348494","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Impact of on-demand connection management in MPI over VIA MPI中按需连接管理对VIA的影响
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137740
Jiesheng Wu, Jiuxing Liu, P. Wyckoff, D. Panda
Designing scalable and efficient Message Passing Interface (MPI) implementations for emerging cluster interconnects such as VIA-based networks and InfiniBand is important for building next generation clusters. In this paper, we address the scalability issue in implementation of MPI over VIA by an on-demand connection management mechanism. On-demand connection management is designed to limit the use of resources by applications that absolutely require them. We address design issues of incorporating the on-demand connection mechanism into an implementation of MPI over VIA. A complete implementation was done for MVICH over both cLAN VIA and Berkeley VIA. Performance evaluation on a set of microbenchmarks and NAS parallel benchmarks demonstrates that the on-demand mechanism can increase the scalability of MPI implementations by limiting the use of resources as needed by applications. It also shows that the on-demand mechanism delivers comparable or better performance as the static mechanism in which a fully-connected process model usually exists in the MPI implementations. These results demonstrate that the on-demand connection mechanism is a feasible solution to increase the scalability of MPI implementations over VIA- and InfiniBand-based networks.
为新兴的集群互连(如基于via的网络和InfiniBand)设计可扩展且高效的消息传递接口(MPI)实现对于构建下一代集群非常重要。在本文中,我们通过按需连接管理机制解决了通过VIA实现MPI的可扩展性问题。按需连接管理旨在限制绝对需要资源的应用程序对资源的使用。我们解决了将按需连接机制整合到通过VIA实现的MPI中的设计问题。在cLAN VIA和Berkeley VIA上完成了MVICH的完整实现。对一组微基准测试和NAS并行基准测试的性能评估表明,按需机制可以根据应用程序的需要限制资源的使用,从而提高MPI实现的可伸缩性。它还表明,随需应变机制提供了与静态机制相当或更好的性能,在静态机制中,MPI实现中通常存在完全连接的流程模型。这些结果表明,按需连接机制是一种可行的解决方案,可以在基于VIA和infiniband的网络上提高MPI实现的可扩展性。
{"title":"Impact of on-demand connection management in MPI over VIA","authors":"Jiesheng Wu, Jiuxing Liu, P. Wyckoff, D. Panda","doi":"10.1109/CLUSTR.2002.1137740","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137740","url":null,"abstract":"Designing scalable and efficient Message Passing Interface (MPI) implementations for emerging cluster interconnects such as VIA-based networks and InfiniBand is important for building next generation clusters. In this paper, we address the scalability issue in implementation of MPI over VIA by an on-demand connection management mechanism. On-demand connection management is designed to limit the use of resources by applications that absolutely require them. We address design issues of incorporating the on-demand connection mechanism into an implementation of MPI over VIA. A complete implementation was done for MVICH over both cLAN VIA and Berkeley VIA. Performance evaluation on a set of microbenchmarks and NAS parallel benchmarks demonstrates that the on-demand mechanism can increase the scalability of MPI implementations by limiting the use of resources as needed by applications. It also shows that the on-demand mechanism delivers comparable or better performance as the static mechanism in which a fully-connected process model usually exists in the MPI implementations. These results demonstrate that the on-demand connection mechanism is a feasible solution to increase the scalability of MPI implementations over VIA- and InfiniBand-based networks.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83743200","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 23
Compiling tiled iteration spaces for clusters 为集群编译平铺迭代空间
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137768
G. Goumas, Nikolaos Drosinos, Maria Athanasaki, N. Koziris
We present a complete end-to-end framework to generate automatic message-passing code for tiled iteration spaces. We consider general parallelepiped tiling transformations and general convex iteration spaces. We aim to address all problems concerning data parallel code generation efficiently by transforming the initial non-rectangular tile to a rectangular one. In this way, data distribution and communication become simple and straightforward. We have implemented our parallelizing techniques in a tool which automatically generates MPI code and run several experiments on a cluster of PCs. Our experimental results show the merit of general parallelepiped tiling transformations, and confirm previous theoretical work on scheduling-optimal tile shapes.
我们提出了一个完整的端到端框架来为平铺迭代空间生成自动消息传递代码。我们考虑了一般的平行六面体平铺变换和一般的凸迭代空间。我们的目标是通过将初始的非矩形贴图转换为矩形贴图来有效地解决所有有关数据并行代码生成的问题。通过这种方式,数据分发和通信变得简单直接。我们已经在一个工具中实现了我们的并行化技术,该工具可以自动生成MPI代码并在pc集群上运行几个实验。实验结果表明了一般平行六面体平铺变换的优点,并证实了以往关于调度最优平铺形状的理论工作。
{"title":"Compiling tiled iteration spaces for clusters","authors":"G. Goumas, Nikolaos Drosinos, Maria Athanasaki, N. Koziris","doi":"10.1109/CLUSTR.2002.1137768","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137768","url":null,"abstract":"We present a complete end-to-end framework to generate automatic message-passing code for tiled iteration spaces. We consider general parallelepiped tiling transformations and general convex iteration spaces. We aim to address all problems concerning data parallel code generation efficiently by transforming the initial non-rectangular tile to a rectangular one. In this way, data distribution and communication become simple and straightforward. We have implemented our parallelizing techniques in a tool which automatically generates MPI code and run several experiments on a cluster of PCs. Our experimental results show the merit of general parallelepiped tiling transformations, and confirm previous theoretical work on scheduling-optimal tile shapes.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72978622","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 16
Leveraging standard core technologies to programmatically build Linux cluster appliances 利用标准核心技术以编程方式构建Linux集群设备
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137728
M. Katz, P. Papadopoulos, Greg Bruno
Clusters have made the jump from lab prototypes to full-fledged production computing platforms. The number variety, and specialized configurations of these machines are increasing dramatically with 32-128 node clusters being commonplace in science labs. The evolving nature of the platform is to target generic PC hardware to specialized functions such as login, compute, Web server file server and a visualization engine. This is the logical extension to the standard login/compute dichotomy of traditional Beowulf clusters. Clearly, these specialized nodes (henceforth "cluster appliances") share an immense amount of common configuration and software. What is lacking in many clustering toolkits is the ability to share configuration across appliances and specific hardware (where it should be shared) and differentiate only where needed In the NPACI Rocks cluster distribution, we have developed a configuration infrastructure with well-defined inheritance properties that leverages and builds on de facto standards including: XML (with standard parsers), RedHat Kickstart, HTTP transport, CGI, SQL databases, and graph constructs to easily define cluster appliances. Our approach neither resorts to replication of configuration files nor does it require building a "golden" image reference. By relying on this descriptive and programmatic infrastructure and carefully demarking configuration information from the software packages (which is a bit delivery mechanism), we can easily handle the heterogeneity of appliances, easily deal with small hardware differences among particular instances of appliances (such as IDE vs. SCSI), and support large hardware differences (like /spl times/86 vs. IA64) with the same infrastructure. Our mechanism is easily extended to other descriptive infrastructures (such as Solaris Jumpstart as a backend target) and has been proven on over a 100 clusters (with significant hardware and configuration differences among these clusters).
集群已经实现了从实验室原型到成熟的生产计算平台的飞跃。随着32-128节点集群在科学实验室中的普及,这些机器的数量变化和专门配置正在急剧增加。该平台的发展本质是将通用PC硬件定位为专门的功能,如登录、计算、Web服务器、文件服务器和可视化引擎。这是对传统Beowulf集群的标准登录/计算二分法的逻辑扩展。显然,这些专门的节点(因此称为“集群设备”)共享大量的公共配置和软件。许多集群工具包缺乏的是跨设备和特定硬件共享配置的能力(应该共享的地方),并且只在需要的地方进行区分。在NPACI Rocks集群发行版中,我们开发了一个具有定义良好的继承属性的配置基础设施,它利用并构建了事实上的标准,包括:XML(带有标准解析器)、RedHat Kickstart、HTTP传输、CGI、SQL数据库和图形结构,以便轻松定义集群设备。我们的方法既不需要复制配置文件,也不需要构建“黄金”映像参考。通过依赖这种描述性和可编程的基础设施,并仔细地从软件包中划分配置信息(这是一种位传递机制),我们可以轻松地处理设备的异构性,轻松处理设备特定实例之间的小硬件差异(例如IDE与SCSI),并使用相同的基础设施支持大硬件差异(例如/spl times/86与IA64)。我们的机制很容易扩展到其他描述性基础设施(例如Solaris Jumpstart作为后端目标),并且已经在100多个集群上得到了验证(这些集群之间存在显著的硬件和配置差异)。
{"title":"Leveraging standard core technologies to programmatically build Linux cluster appliances","authors":"M. Katz, P. Papadopoulos, Greg Bruno","doi":"10.1109/CLUSTR.2002.1137728","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137728","url":null,"abstract":"Clusters have made the jump from lab prototypes to full-fledged production computing platforms. The number variety, and specialized configurations of these machines are increasing dramatically with 32-128 node clusters being commonplace in science labs. The evolving nature of the platform is to target generic PC hardware to specialized functions such as login, compute, Web server file server and a visualization engine. This is the logical extension to the standard login/compute dichotomy of traditional Beowulf clusters. Clearly, these specialized nodes (henceforth \"cluster appliances\") share an immense amount of common configuration and software. What is lacking in many clustering toolkits is the ability to share configuration across appliances and specific hardware (where it should be shared) and differentiate only where needed In the NPACI Rocks cluster distribution, we have developed a configuration infrastructure with well-defined inheritance properties that leverages and builds on de facto standards including: XML (with standard parsers), RedHat Kickstart, HTTP transport, CGI, SQL databases, and graph constructs to easily define cluster appliances. Our approach neither resorts to replication of configuration files nor does it require building a \"golden\" image reference. By relying on this descriptive and programmatic infrastructure and carefully demarking configuration information from the software packages (which is a bit delivery mechanism), we can easily handle the heterogeneity of appliances, easily deal with small hardware differences among particular instances of appliances (such as IDE vs. SCSI), and support large hardware differences (like /spl times/86 vs. IA64) with the same infrastructure. Our mechanism is easily extended to other descriptive infrastructures (such as Solaris Jumpstart as a backend target) and has been proven on over a 100 clusters (with significant hardware and configuration differences among these clusters).","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84400697","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 30
Experience in offloading protocol processing to a programmable NIC 有将协议处理卸载到可编程网卡的经验
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137730
A. Maccabe, Wenbin Zhu, J. Otto, R. Riesen
Offloading protocol processing will become an important tool in supporting our efforts to deliver increasing bandwidth to applications. In this paper we describe our experience in offloading protocol processing to a programmable gigabit Ethernet network interface card. For our experiments, we selected a simple RTS/CTS (request to send/clear to send) protocol called RMPP (reliable message passing protocol). This protocol provides end-to-end flow control and full message retransmit in the case of a lost or corrupt packet. By carefully selecting parts of the protocol for offloading, we were able to improve the bandwidth delivered to MPI applications from approximately 280 Mb/s to approximately 700 Mb/s using standard, 1500 byte, Ethernet frames. Using "jumbo", 9000 byte frames the bandwidth improves from approximately 425 Mb/s to 840 Mb/s. Moreover, we were able to show a significant increase in the availability of the host processor.
卸载协议处理将成为支持我们向应用程序提供不断增长的带宽的重要工具。本文描述了我们在将协议处理卸载到可编程千兆以太网网卡上的经验。在我们的实验中,我们选择了一个简单的RTS/CTS(请求发送/清除发送)协议,称为RMPP(可靠消息传递协议)。该协议提供端到端流控制,在丢失或损坏数据包的情况下提供完整的消息重传。通过仔细选择用于卸载的协议部分,我们能够使用标准的1500字节以太网帧将传输到MPI应用程序的带宽从大约280 Mb/s提高到大约700 Mb/s。使用“jumbo”,即9000字节帧,带宽从大约425 Mb/s提高到840 Mb/s。此外,我们还展示了主机处理器可用性的显著提高。
{"title":"Experience in offloading protocol processing to a programmable NIC","authors":"A. Maccabe, Wenbin Zhu, J. Otto, R. Riesen","doi":"10.1109/CLUSTR.2002.1137730","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137730","url":null,"abstract":"Offloading protocol processing will become an important tool in supporting our efforts to deliver increasing bandwidth to applications. In this paper we describe our experience in offloading protocol processing to a programmable gigabit Ethernet network interface card. For our experiments, we selected a simple RTS/CTS (request to send/clear to send) protocol called RMPP (reliable message passing protocol). This protocol provides end-to-end flow control and full message retransmit in the case of a lost or corrupt packet. By carefully selecting parts of the protocol for offloading, we were able to improve the bandwidth delivered to MPI applications from approximately 280 Mb/s to approximately 700 Mb/s using standard, 1500 byte, Ethernet frames. Using \"jumbo\", 9000 byte frames the bandwidth improves from approximately 425 Mb/s to 840 Mb/s. Moreover, we were able to show a significant increase in the availability of the host processor.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81502172","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 26
Scalable loop self-scheduling schemes for heterogeneous clusters 异构集群的可伸缩循环自调度方案
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137767
Anthony T. Chronopoulos, Satish Penmatsa, Ning Yu
Distributed systems (e.g. a LAN of computers) can be used for concurrent processing for some applications. However a serious difficulty in concurrent programming of a distributed system is how to deal with scheduling and load balancing of such a system which may consist of heterogeneous computers. Distributed scheduling schemes suitable for parallel loops with independent iterations on heterogeneous computer clusters have been proposed and analyzed in the past. Here, we implement the previous schemes in CORBA (Orbix). We also present an extension of these schemes implemented in a hierarchical master-slave architecture. We present experimental results and comparisons.
分布式系统(例如计算机局域网)可以用于某些应用程序的并发处理。然而,分布式系统并发编程的一个严重困难是如何处理这种可能由异构计算机组成的系统的调度和负载平衡。过去已经提出并分析了适用于异构计算机集群上具有独立迭代的并行循环的分布式调度方案。这里,我们在CORBA (Orbix)中实现前面的方案。我们还提出了在分层主从架构中实现这些方案的扩展。我们给出了实验结果和比较。
{"title":"Scalable loop self-scheduling schemes for heterogeneous clusters","authors":"Anthony T. Chronopoulos, Satish Penmatsa, Ning Yu","doi":"10.1109/CLUSTR.2002.1137767","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137767","url":null,"abstract":"Distributed systems (e.g. a LAN of computers) can be used for concurrent processing for some applications. However a serious difficulty in concurrent programming of a distributed system is how to deal with scheduling and load balancing of such a system which may consist of heterogeneous computers. Distributed scheduling schemes suitable for parallel loops with independent iterations on heterogeneous computer clusters have been proposed and analyzed in the past. Here, we implement the previous schemes in CORBA (Orbix). We also present an extension of these schemes implemented in a hierarchical master-slave architecture. We present experimental results and comparisons.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83783070","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 32
期刊
Proceedings. IEEE International Conference on Cluster Computing
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1