首页 > 最新文献

Proceedings. IEEE International Conference on Cluster Computing最新文献

英文 中文
On the evaluation of JavaSymphony for cluster applications 关于集群应用中JavaSymphony的评价
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137772
T. Fahringer, A. Jugravu, B. D. Martino, S. Venticinque, H. Moritsch
In the past few years, increasing interest has been shown in using Java as a language for performance-oriented distributed and parallel computing. Most Java-based systems that support portable parallel and distributed computing either require the programmer to deal with intricate low level details of Java which can be a tedious, time-consuming and error-prone task, or prevent the programmer from controlling locality of data. In contrast to most existing systems, JavaSymphony - a class library written entirely in Java - allows to control parallelism, load balancing and locality at a high level. Objects can be explicitly distributed and migrated based on virtual architectures which impose a virtual hierarchy on a distributed/parallel system of physical computing nodes. The concept of blocking/nonblocking remote method invocation is used to exchange data among distributed objects and to process work by remote objects. We evaluate the JavaSymphony programming API for a variety of distributed/parallel algorithms which comprises backtracking, N-body, encryption/decryption algorithms and asynchronous nested optimization algorithms. Performance results are presented for both homogeneous and heterogeneous cluster architectures. Moreover, we compare JavaSymphony with an alternative well-known semi-automatic system.
在过去的几年中,越来越多的人对使用Java作为面向性能的分布式和并行计算语言感兴趣。大多数支持可移植并行和分布式计算的基于Java的系统,要么要求程序员处理复杂的Java底层细节,这可能是一项乏味、耗时且容易出错的任务,要么阻止程序员控制数据的位置。与大多数现有系统相比,JavaSymphony——一个完全用Java编写的类库——允许在高层次上控制并行性、负载平衡和局部性。对象可以基于虚拟架构进行显式分布和迁移,虚拟架构在物理计算节点的分布式/并行系统上施加了虚拟层次结构。阻塞/非阻塞远程方法调用的概念用于在分布式对象之间交换数据和处理远程对象的工作。我们评估了各种分布式/并行算法的JavaSymphony编程API,包括回溯,N-body,加密/解密算法和异步嵌套优化算法。给出了同构和异构集群体系结构的性能结果。此外,我们还将JavaSymphony与另一个知名的半自动系统进行了比较。
{"title":"On the evaluation of JavaSymphony for cluster applications","authors":"T. Fahringer, A. Jugravu, B. D. Martino, S. Venticinque, H. Moritsch","doi":"10.1109/CLUSTR.2002.1137772","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137772","url":null,"abstract":"In the past few years, increasing interest has been shown in using Java as a language for performance-oriented distributed and parallel computing. Most Java-based systems that support portable parallel and distributed computing either require the programmer to deal with intricate low level details of Java which can be a tedious, time-consuming and error-prone task, or prevent the programmer from controlling locality of data. In contrast to most existing systems, JavaSymphony - a class library written entirely in Java - allows to control parallelism, load balancing and locality at a high level. Objects can be explicitly distributed and migrated based on virtual architectures which impose a virtual hierarchy on a distributed/parallel system of physical computing nodes. The concept of blocking/nonblocking remote method invocation is used to exchange data among distributed objects and to process work by remote objects. We evaluate the JavaSymphony programming API for a variety of distributed/parallel algorithms which comprises backtracking, N-body, encryption/decryption algorithms and asynchronous nested optimization algorithms. Performance results are presented for both homogeneous and heterogeneous cluster architectures. Moreover, we compare JavaSymphony with an alternative well-known semi-automatic system.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":"1 1","pages":"394-401"},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89953505","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
Trends in high performance computing and using numerical libraries on clusters 高性能计算和在集群上使用数字库的趋势
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137743
J. Dongarra
In this talk we will look at how High Performance computing has changed over the last 10-year and look toward the future in terms of trends with a focus on cluster computing. We will also looks at an approach for deploying numerical libraries on clusters, called LAPACK for Clusters (LFC). The LFC software intends to allow users to dynamically link against an archived library of executable routines. The user is assumed to call one of the LFC routines from a single processor on the cluster. The intent is to possibly leverage the parallel computing power of the cluster to solve the problem on the user’s behalf. The software accounts for the details required for parallelizing the user’s problem such as resource discovery and selection, and mapping the data onto and off of the process grid in addition to executing the parallel library routine itself.
在本次演讲中,我们将介绍高性能计算在过去十年中的变化,并以集群计算为重点,展望高性能计算的未来趋势。我们还将研究一种在集群上部署数字库的方法,称为集群的LAPACK (LFC)。LFC软件旨在允许用户动态链接可执行例程的存档库。假设用户从集群上的单个处理器调用LFC例程之一。其目的是尽可能利用集群的并行计算能力来代表用户解决问题。除了执行并行库例程本身外,软件还考虑了并行化用户问题所需的细节,例如资源发现和选择,以及将数据映射到进程网格上和网格外。
{"title":"Trends in high performance computing and using numerical libraries on clusters","authors":"J. Dongarra","doi":"10.1109/CLUSTR.2002.1137743","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137743","url":null,"abstract":"In this talk we will look at how High Performance computing has changed over the last 10-year and look toward the future in terms of trends with a focus on cluster computing. We will also looks at an approach for deploying numerical libraries on clusters, called LAPACK for Clusters (LFC). The LFC software intends to allow users to dynamically link against an archived library of executable routines. The user is assumed to call one of the LFC routines from a single processor on the cluster. The intent is to possibly leverage the parallel computing power of the cluster to solve the problem on the user’s behalf. The software accounts for the details required for parallelizing the user’s problem such as resource discovery and selection, and mapping the data onto and off of the process grid in addition to executing the parallel library routine itself.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":"142 1","pages":"172-172"},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86639138","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Design and implementation of CC-NUMA card II for SCI-based PC clustering 基于sci的PC机集群CC-NUMA卡II的设计与实现
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137739
Soo-Cheol Oh, Sang-Hwa Chung, Hankook Jang
It is extremely important to minimize network access time when constructing a high-performance PC cluster system. For an SCI-based PC cluster it is possible to reduce the network access time by maintaining network cache in each cluster node. This paper presents the second version CC-NUMA card (CC-NUMA card II) that utilizes network cache for SCI-based PC clustering. The CC-NUMA card II is directly plugged into the PCI slot of each node, and contains shared memory, network cache, a shared memory control module and network control module. The network cache is maintained for shared memory on the PCI bus of cluster nodes. The coherency mechanism between network cache and shared memory is based on the IEEE SCI standard. In previous research, the first version CC-NUMA card (CC-NUMA card I) was developed. The CC-NUMA card I adopting Dolphin's PCI-SCI card as the network control module caused overhead in exchanging data between the remote nodes. In this paper, the overhead is removed by developing the CC-NUMA card II that combines the shared memory control module and network control module in a single card. Throughout the experiment with the SPLASH-2 benchmark suite, the CC-NUMA card II based PC cluster shows better performance than a NUMA system based on Dolphin's PCI-SCI card.
在构建高性能PC集群系统时,最小化网络访问时间是非常重要的。对于基于sci的PC集群,可以通过在每个集群节点中维护网络缓存来减少网络访问时间。本文提出了第二版CC-NUMA卡(CC-NUMA卡II),它利用网络缓存实现基于sci的PC机集群。CC-NUMA卡II直接插入每个节点的PCI插槽,包含共享内存、网络缓存、共享内存控制模块和网络控制模块。为集群节点的PCI总线上的共享内存维护网络缓存。网络缓存与共享内存之间的一致性机制基于IEEE SCI标准。在以往的研究中,开发了第一个版本的CC-NUMA卡(CC-NUMA卡I)。采用Dolphin的PCI-SCI卡作为网络控制模块的CC-NUMA卡I造成了远程节点间数据交换的开销。本文通过开发将共享内存控制模块和网络控制模块结合在一块卡中的CC-NUMA卡II来消除开销。在splash2基准测试套件的整个实验中,基于CC-NUMA卡II的PC集群表现出比基于Dolphin的PCI-SCI卡的NUMA系统更好的性能。
{"title":"Design and implementation of CC-NUMA card II for SCI-based PC clustering","authors":"Soo-Cheol Oh, Sang-Hwa Chung, Hankook Jang","doi":"10.1109/CLUSTR.2002.1137739","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137739","url":null,"abstract":"It is extremely important to minimize network access time when constructing a high-performance PC cluster system. For an SCI-based PC cluster it is possible to reduce the network access time by maintaining network cache in each cluster node. This paper presents the second version CC-NUMA card (CC-NUMA card II) that utilizes network cache for SCI-based PC clustering. The CC-NUMA card II is directly plugged into the PCI slot of each node, and contains shared memory, network cache, a shared memory control module and network control module. The network cache is maintained for shared memory on the PCI bus of cluster nodes. The coherency mechanism between network cache and shared memory is based on the IEEE SCI standard. In previous research, the first version CC-NUMA card (CC-NUMA card I) was developed. The CC-NUMA card I adopting Dolphin's PCI-SCI card as the network control module caused overhead in exchanging data between the remote nodes. In this paper, the overhead is removed by developing the CC-NUMA card II that combines the shared memory control module and network control module in a single card. Throughout the experiment with the SPLASH-2 benchmark suite, the CC-NUMA card II based PC cluster shows better performance than a NUMA system based on Dolphin's PCI-SCI card.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":"1 1","pages":"145-151"},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84551825","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Clusters as large-scale development facilities 作为大型开发设施的集群
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137729
R. Evard, N. Desai, J. Navarro, Daniel Nurmi
In this paper, we describe the use of a cluster as a generalized facility for development. A development facility is a system used primarily for testing and development activities while being operated reliably for many users. We are in the midst of a project to build and operate a large-scale development facility. We discuss our motivation for using clusters in this way and compare the model with a classic computing facility. We describe our experiences and findings from the first phase of this project. Many of these observations are relevant to the design of standard clusters and to future development facilities.
在本文中,我们描述了集群作为一种通用的开发工具的使用。开发设施是主要用于测试和开发活动的系统,同时可为许多用户可靠地操作。我们正在进行一个建设和运营大型开发设施的项目。我们讨论了以这种方式使用集群的动机,并将该模型与经典计算工具进行了比较。我们描述了我们在这个项目第一阶段的经验和发现。这些观察结果中的许多都与标准集群的设计和未来的开发设施有关。
{"title":"Clusters as large-scale development facilities","authors":"R. Evard, N. Desai, J. Navarro, Daniel Nurmi","doi":"10.1109/CLUSTR.2002.1137729","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137729","url":null,"abstract":"In this paper, we describe the use of a cluster as a generalized facility for development. A development facility is a system used primarily for testing and development activities while being operated reliably for many users. We are in the midst of a project to build and operate a large-scale development facility. We discuss our motivation for using clusters in this way and compare the model with a classic computing facility. We describe our experiences and findings from the first phase of this project. Many of these observations are relevant to the design of standard clusters and to future development facilities.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":"31 1","pages":"54-63"},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91214538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
Design of a middleware-based cluster management platform with task management and migration 基于中间件的集群管理平台,具有任务管理和迁移功能
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137788
F. Turck, S. Vanhastel, P. Thysebaert, B. Volckaert, P. Demeester, B. Dhoedt
In this paper, we address the design and implementation of a generic and scalable platform for efficient management of computational resources. The developed platform is called the Intelligent Agent Platform. Its architecture is based on middleware technology in order to ensure easy distribution of the software components between the participating workstations and to exploit advanced software techniques. The computational tasks are referred to as agents, defined as software components that are capable of executing particular algorithms on input data. The platform offers advanced features such as transparent task management, load balancing, run time compilation of agent code and task migration and is therefore denoted by the adjective "Intelligent". The architecture of the platform will be outlined from a computational point of view and each component will be described in detail. Furthermore, some important design issues of the platform are covered and a performance evaluation is presented.
在本文中,我们讨论了一个通用的、可扩展的平台的设计和实现,用于有效地管理计算资源。开发的平台称为智能代理平台。它的体系结构基于中间件技术,以确保软件组件在参与的工作站之间易于分发,并利用先进的软件技术。计算任务被称为代理,定义为能够对输入数据执行特定算法的软件组件。该平台提供了一些高级特性,如透明的任务管理、负载平衡、代理代码的运行时编译和任务迁移,因此被称为“智能”。将从计算的角度概述平台的体系结构,并详细描述每个组件。此外,还讨论了该平台的一些重要设计问题,并给出了性能评价。
{"title":"Design of a middleware-based cluster management platform with task management and migration","authors":"F. Turck, S. Vanhastel, P. Thysebaert, B. Volckaert, P. Demeester, B. Dhoedt","doi":"10.1109/CLUSTR.2002.1137788","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137788","url":null,"abstract":"In this paper, we address the design and implementation of a generic and scalable platform for efficient management of computational resources. The developed platform is called the Intelligent Agent Platform. Its architecture is based on middleware technology in order to ensure easy distribution of the software components between the participating workstations and to exploit advanced software techniques. The computational tasks are referred to as agents, defined as software components that are capable of executing particular algorithms on input data. The platform offers advanced features such as transparent task management, load balancing, run time compilation of agent code and task migration and is therefore denoted by the adjective \"Intelligent\". The architecture of the platform will be outlined from a computational point of view and each component will be described in detail. Furthermore, some important design issues of the platform are covered and a performance evaluation is presented.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":"3 1","pages":"484-487"},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83348494","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Impact of on-demand connection management in MPI over VIA MPI中按需连接管理对VIA的影响
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137740
Jiesheng Wu, Jiuxing Liu, P. Wyckoff, D. Panda
Designing scalable and efficient Message Passing Interface (MPI) implementations for emerging cluster interconnects such as VIA-based networks and InfiniBand is important for building next generation clusters. In this paper, we address the scalability issue in implementation of MPI over VIA by an on-demand connection management mechanism. On-demand connection management is designed to limit the use of resources by applications that absolutely require them. We address design issues of incorporating the on-demand connection mechanism into an implementation of MPI over VIA. A complete implementation was done for MVICH over both cLAN VIA and Berkeley VIA. Performance evaluation on a set of microbenchmarks and NAS parallel benchmarks demonstrates that the on-demand mechanism can increase the scalability of MPI implementations by limiting the use of resources as needed by applications. It also shows that the on-demand mechanism delivers comparable or better performance as the static mechanism in which a fully-connected process model usually exists in the MPI implementations. These results demonstrate that the on-demand connection mechanism is a feasible solution to increase the scalability of MPI implementations over VIA- and InfiniBand-based networks.
为新兴的集群互连(如基于via的网络和InfiniBand)设计可扩展且高效的消息传递接口(MPI)实现对于构建下一代集群非常重要。在本文中,我们通过按需连接管理机制解决了通过VIA实现MPI的可扩展性问题。按需连接管理旨在限制绝对需要资源的应用程序对资源的使用。我们解决了将按需连接机制整合到通过VIA实现的MPI中的设计问题。在cLAN VIA和Berkeley VIA上完成了MVICH的完整实现。对一组微基准测试和NAS并行基准测试的性能评估表明,按需机制可以根据应用程序的需要限制资源的使用,从而提高MPI实现的可伸缩性。它还表明,随需应变机制提供了与静态机制相当或更好的性能,在静态机制中,MPI实现中通常存在完全连接的流程模型。这些结果表明,按需连接机制是一种可行的解决方案,可以在基于VIA和infiniband的网络上提高MPI实现的可扩展性。
{"title":"Impact of on-demand connection management in MPI over VIA","authors":"Jiesheng Wu, Jiuxing Liu, P. Wyckoff, D. Panda","doi":"10.1109/CLUSTR.2002.1137740","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137740","url":null,"abstract":"Designing scalable and efficient Message Passing Interface (MPI) implementations for emerging cluster interconnects such as VIA-based networks and InfiniBand is important for building next generation clusters. In this paper, we address the scalability issue in implementation of MPI over VIA by an on-demand connection management mechanism. On-demand connection management is designed to limit the use of resources by applications that absolutely require them. We address design issues of incorporating the on-demand connection mechanism into an implementation of MPI over VIA. A complete implementation was done for MVICH over both cLAN VIA and Berkeley VIA. Performance evaluation on a set of microbenchmarks and NAS parallel benchmarks demonstrates that the on-demand mechanism can increase the scalability of MPI implementations by limiting the use of resources as needed by applications. It also shows that the on-demand mechanism delivers comparable or better performance as the static mechanism in which a fully-connected process model usually exists in the MPI implementations. These results demonstrate that the on-demand connection mechanism is a feasible solution to increase the scalability of MPI implementations over VIA- and InfiniBand-based networks.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":"29 1","pages":"152-159"},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83743200","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 23
Compiling tiled iteration spaces for clusters 为集群编译平铺迭代空间
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137768
G. Goumas, Nikolaos Drosinos, Maria Athanasaki, N. Koziris
We present a complete end-to-end framework to generate automatic message-passing code for tiled iteration spaces. We consider general parallelepiped tiling transformations and general convex iteration spaces. We aim to address all problems concerning data parallel code generation efficiently by transforming the initial non-rectangular tile to a rectangular one. In this way, data distribution and communication become simple and straightforward. We have implemented our parallelizing techniques in a tool which automatically generates MPI code and run several experiments on a cluster of PCs. Our experimental results show the merit of general parallelepiped tiling transformations, and confirm previous theoretical work on scheduling-optimal tile shapes.
我们提出了一个完整的端到端框架来为平铺迭代空间生成自动消息传递代码。我们考虑了一般的平行六面体平铺变换和一般的凸迭代空间。我们的目标是通过将初始的非矩形贴图转换为矩形贴图来有效地解决所有有关数据并行代码生成的问题。通过这种方式,数据分发和通信变得简单直接。我们已经在一个工具中实现了我们的并行化技术,该工具可以自动生成MPI代码并在pc集群上运行几个实验。实验结果表明了一般平行六面体平铺变换的优点,并证实了以往关于调度最优平铺形状的理论工作。
{"title":"Compiling tiled iteration spaces for clusters","authors":"G. Goumas, Nikolaos Drosinos, Maria Athanasaki, N. Koziris","doi":"10.1109/CLUSTR.2002.1137768","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137768","url":null,"abstract":"We present a complete end-to-end framework to generate automatic message-passing code for tiled iteration spaces. We consider general parallelepiped tiling transformations and general convex iteration spaces. We aim to address all problems concerning data parallel code generation efficiently by transforming the initial non-rectangular tile to a rectangular one. In this way, data distribution and communication become simple and straightforward. We have implemented our parallelizing techniques in a tool which automatically generates MPI code and run several experiments on a cluster of PCs. Our experimental results show the merit of general parallelepiped tiling transformations, and confirm previous theoretical work on scheduling-optimal tile shapes.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":"98 1","pages":"360-369"},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72978622","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 16
Leveraging standard core technologies to programmatically build Linux cluster appliances 利用标准核心技术以编程方式构建Linux集群设备
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137728
M. Katz, P. Papadopoulos, Greg Bruno
Clusters have made the jump from lab prototypes to full-fledged production computing platforms. The number variety, and specialized configurations of these machines are increasing dramatically with 32-128 node clusters being commonplace in science labs. The evolving nature of the platform is to target generic PC hardware to specialized functions such as login, compute, Web server file server and a visualization engine. This is the logical extension to the standard login/compute dichotomy of traditional Beowulf clusters. Clearly, these specialized nodes (henceforth "cluster appliances") share an immense amount of common configuration and software. What is lacking in many clustering toolkits is the ability to share configuration across appliances and specific hardware (where it should be shared) and differentiate only where needed In the NPACI Rocks cluster distribution, we have developed a configuration infrastructure with well-defined inheritance properties that leverages and builds on de facto standards including: XML (with standard parsers), RedHat Kickstart, HTTP transport, CGI, SQL databases, and graph constructs to easily define cluster appliances. Our approach neither resorts to replication of configuration files nor does it require building a "golden" image reference. By relying on this descriptive and programmatic infrastructure and carefully demarking configuration information from the software packages (which is a bit delivery mechanism), we can easily handle the heterogeneity of appliances, easily deal with small hardware differences among particular instances of appliances (such as IDE vs. SCSI), and support large hardware differences (like /spl times/86 vs. IA64) with the same infrastructure. Our mechanism is easily extended to other descriptive infrastructures (such as Solaris Jumpstart as a backend target) and has been proven on over a 100 clusters (with significant hardware and configuration differences among these clusters).
集群已经实现了从实验室原型到成熟的生产计算平台的飞跃。随着32-128节点集群在科学实验室中的普及,这些机器的数量变化和专门配置正在急剧增加。该平台的发展本质是将通用PC硬件定位为专门的功能,如登录、计算、Web服务器、文件服务器和可视化引擎。这是对传统Beowulf集群的标准登录/计算二分法的逻辑扩展。显然,这些专门的节点(因此称为“集群设备”)共享大量的公共配置和软件。许多集群工具包缺乏的是跨设备和特定硬件共享配置的能力(应该共享的地方),并且只在需要的地方进行区分。在NPACI Rocks集群发行版中,我们开发了一个具有定义良好的继承属性的配置基础设施,它利用并构建了事实上的标准,包括:XML(带有标准解析器)、RedHat Kickstart、HTTP传输、CGI、SQL数据库和图形结构,以便轻松定义集群设备。我们的方法既不需要复制配置文件,也不需要构建“黄金”映像参考。通过依赖这种描述性和可编程的基础设施,并仔细地从软件包中划分配置信息(这是一种位传递机制),我们可以轻松地处理设备的异构性,轻松处理设备特定实例之间的小硬件差异(例如IDE与SCSI),并使用相同的基础设施支持大硬件差异(例如/spl times/86与IA64)。我们的机制很容易扩展到其他描述性基础设施(例如Solaris Jumpstart作为后端目标),并且已经在100多个集群上得到了验证(这些集群之间存在显著的硬件和配置差异)。
{"title":"Leveraging standard core technologies to programmatically build Linux cluster appliances","authors":"M. Katz, P. Papadopoulos, Greg Bruno","doi":"10.1109/CLUSTR.2002.1137728","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137728","url":null,"abstract":"Clusters have made the jump from lab prototypes to full-fledged production computing platforms. The number variety, and specialized configurations of these machines are increasing dramatically with 32-128 node clusters being commonplace in science labs. The evolving nature of the platform is to target generic PC hardware to specialized functions such as login, compute, Web server file server and a visualization engine. This is the logical extension to the standard login/compute dichotomy of traditional Beowulf clusters. Clearly, these specialized nodes (henceforth \"cluster appliances\") share an immense amount of common configuration and software. What is lacking in many clustering toolkits is the ability to share configuration across appliances and specific hardware (where it should be shared) and differentiate only where needed In the NPACI Rocks cluster distribution, we have developed a configuration infrastructure with well-defined inheritance properties that leverages and builds on de facto standards including: XML (with standard parsers), RedHat Kickstart, HTTP transport, CGI, SQL databases, and graph constructs to easily define cluster appliances. Our approach neither resorts to replication of configuration files nor does it require building a \"golden\" image reference. By relying on this descriptive and programmatic infrastructure and carefully demarking configuration information from the software packages (which is a bit delivery mechanism), we can easily handle the heterogeneity of appliances, easily deal with small hardware differences among particular instances of appliances (such as IDE vs. SCSI), and support large hardware differences (like /spl times/86 vs. IA64) with the same infrastructure. Our mechanism is easily extended to other descriptive infrastructures (such as Solaris Jumpstart as a backend target) and has been proven on over a 100 clusters (with significant hardware and configuration differences among these clusters).","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":"32 1","pages":"47-53"},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84400697","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 30
Experience in offloading protocol processing to a programmable NIC 有将协议处理卸载到可编程网卡的经验
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137730
A. Maccabe, Wenbin Zhu, J. Otto, R. Riesen
Offloading protocol processing will become an important tool in supporting our efforts to deliver increasing bandwidth to applications. In this paper we describe our experience in offloading protocol processing to a programmable gigabit Ethernet network interface card. For our experiments, we selected a simple RTS/CTS (request to send/clear to send) protocol called RMPP (reliable message passing protocol). This protocol provides end-to-end flow control and full message retransmit in the case of a lost or corrupt packet. By carefully selecting parts of the protocol for offloading, we were able to improve the bandwidth delivered to MPI applications from approximately 280 Mb/s to approximately 700 Mb/s using standard, 1500 byte, Ethernet frames. Using "jumbo", 9000 byte frames the bandwidth improves from approximately 425 Mb/s to 840 Mb/s. Moreover, we were able to show a significant increase in the availability of the host processor.
卸载协议处理将成为支持我们向应用程序提供不断增长的带宽的重要工具。本文描述了我们在将协议处理卸载到可编程千兆以太网网卡上的经验。在我们的实验中,我们选择了一个简单的RTS/CTS(请求发送/清除发送)协议,称为RMPP(可靠消息传递协议)。该协议提供端到端流控制,在丢失或损坏数据包的情况下提供完整的消息重传。通过仔细选择用于卸载的协议部分,我们能够使用标准的1500字节以太网帧将传输到MPI应用程序的带宽从大约280 Mb/s提高到大约700 Mb/s。使用“jumbo”,即9000字节帧,带宽从大约425 Mb/s提高到840 Mb/s。此外,我们还展示了主机处理器可用性的显著提高。
{"title":"Experience in offloading protocol processing to a programmable NIC","authors":"A. Maccabe, Wenbin Zhu, J. Otto, R. Riesen","doi":"10.1109/CLUSTR.2002.1137730","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137730","url":null,"abstract":"Offloading protocol processing will become an important tool in supporting our efforts to deliver increasing bandwidth to applications. In this paper we describe our experience in offloading protocol processing to a programmable gigabit Ethernet network interface card. For our experiments, we selected a simple RTS/CTS (request to send/clear to send) protocol called RMPP (reliable message passing protocol). This protocol provides end-to-end flow control and full message retransmit in the case of a lost or corrupt packet. By carefully selecting parts of the protocol for offloading, we were able to improve the bandwidth delivered to MPI applications from approximately 280 Mb/s to approximately 700 Mb/s using standard, 1500 byte, Ethernet frames. Using \"jumbo\", 9000 byte frames the bandwidth improves from approximately 425 Mb/s to 840 Mb/s. Moreover, we were able to show a significant increase in the availability of the host processor.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":"65 1","pages":"67-74"},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81502172","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 26
Scalable loop self-scheduling schemes for heterogeneous clusters 异构集群的可伸缩循环自调度方案
Pub Date : 2002-09-23 DOI: 10.1109/CLUSTR.2002.1137767
Anthony T. Chronopoulos, Satish Penmatsa, Ning Yu
Distributed systems (e.g. a LAN of computers) can be used for concurrent processing for some applications. However a serious difficulty in concurrent programming of a distributed system is how to deal with scheduling and load balancing of such a system which may consist of heterogeneous computers. Distributed scheduling schemes suitable for parallel loops with independent iterations on heterogeneous computer clusters have been proposed and analyzed in the past. Here, we implement the previous schemes in CORBA (Orbix). We also present an extension of these schemes implemented in a hierarchical master-slave architecture. We present experimental results and comparisons.
分布式系统(例如计算机局域网)可以用于某些应用程序的并发处理。然而,分布式系统并发编程的一个严重困难是如何处理这种可能由异构计算机组成的系统的调度和负载平衡。过去已经提出并分析了适用于异构计算机集群上具有独立迭代的并行循环的分布式调度方案。这里,我们在CORBA (Orbix)中实现前面的方案。我们还提出了在分层主从架构中实现这些方案的扩展。我们给出了实验结果和比较。
{"title":"Scalable loop self-scheduling schemes for heterogeneous clusters","authors":"Anthony T. Chronopoulos, Satish Penmatsa, Ning Yu","doi":"10.1109/CLUSTR.2002.1137767","DOIUrl":"https://doi.org/10.1109/CLUSTR.2002.1137767","url":null,"abstract":"Distributed systems (e.g. a LAN of computers) can be used for concurrent processing for some applications. However a serious difficulty in concurrent programming of a distributed system is how to deal with scheduling and load balancing of such a system which may consist of heterogeneous computers. Distributed scheduling schemes suitable for parallel loops with independent iterations on heterogeneous computer clusters have been proposed and analyzed in the past. Here, we implement the previous schemes in CORBA (Orbix). We also present an extension of these schemes implemented in a hierarchical master-slave architecture. We present experimental results and comparisons.","PeriodicalId":92128,"journal":{"name":"Proceedings. IEEE International Conference on Cluster Computing","volume":"493 1","pages":"353-359"},"PeriodicalIF":0.0,"publicationDate":"2002-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83783070","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 32
期刊
Proceedings. IEEE International Conference on Cluster Computing
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1