首页 > 最新文献

2008 IEEE International Conference on Cluster Computing最新文献

英文 中文
Reliable adaptable Network RAM 可靠的自适应网络RAM
Pub Date : 2008-10-31 DOI: 10.1109/CLUSTR.2008.4663750
T. Newhall, D. Amato, A. Pshenichkin
We present reliability solutions for adaptable network RAM systems running on general-purpose clusters. Network RAM allows nodes with over-committed memory to swap pages over the network, storing them in the idle RAM of other nodes and avoiding swapping to slow, local disk. An adaptable network RAM system adjusts the amount of RAM currently available for storing remotely swapped pages in response to changes in nodespsila local RAM usage. It is important that network RAM systems provide reliability for remotely swapped page data. Without reliability, a single node failure can result in failure of unrelated processes running on other nodes by losing their remotely swapped pages. Adaptable network RAM systems pose extra difficulties in providing reliability because each nodepsilas capacity for storing remotely swapped pages changes over time, and because pages may move from node to node in response to these changes. Our novel dynamic RAID-based reliability solutions use idle RAM for storing page and reliability data, avoiding using slow disk for reliability. They are designed to work with the adaptive nature of our network RAM system (Nswap), allowing page and reliability data to migrate from node to node and allowing pages to be added to or removed from different parity groups. Additionally, page recovery runs concurrently with cluster applications, so that cluster applications do not have to wait until all data from a failed node is recovered before resuming execution. We present results comparing Nswap to disk swapping for a set of benchmarks running on our gigabit cluster. Our results show that reliable Nswap is up to 32 times faster than swapping to disk, and that there is virtually no impact on the performance of applications as they run concurrently with page recovery.
我们提出了在通用集群上运行的自适应网络RAM系统的可靠性解决方案。网络RAM允许内存过度使用的节点通过网络交换页面,将它们存储在其他节点的空闲RAM中,避免交换到速度较慢的本地磁盘。可适应的网络RAM系统根据节点和本地RAM使用情况的变化,调整当前可用于存储远程交换页面的RAM数量。网络RAM系统为远程交换页数据提供可靠性是很重要的。如果没有可靠性,单个节点的故障可能会导致在其他节点上运行的不相关进程的故障,因为它们会丢失远程交换的页面。适应性网络RAM系统在提供可靠性方面带来了额外的困难,因为每个节点存储远程交换页面的能力会随着时间的推移而变化,而且页面可能会根据这些变化从一个节点移动到另一个节点。我们新颖的基于动态raid的可靠性解决方案使用空闲RAM来存储页面和可靠性数据,避免使用慢速磁盘来实现可靠性。它们被设计为与我们的网络RAM系统(swap)的自适应特性一起工作,允许页面和可靠性数据从节点迁移到节点,并允许将页面添加到不同的奇偶校验组中或从不同的奇偶校验组中删除。此外,页面恢复与集群应用程序并发运行,因此集群应用程序不必等到故障节点的所有数据都恢复后才恢复执行。我们提供了在千兆集群上运行的一组基准测试中比较swap和磁盘交换的结果。我们的结果表明,可靠的swap比交换到磁盘快32倍,并且由于它们与页面恢复并发运行,因此几乎不会影响应用程序的性能。
{"title":"Reliable adaptable Network RAM","authors":"T. Newhall, D. Amato, A. Pshenichkin","doi":"10.1109/CLUSTR.2008.4663750","DOIUrl":"https://doi.org/10.1109/CLUSTR.2008.4663750","url":null,"abstract":"We present reliability solutions for adaptable network RAM systems running on general-purpose clusters. Network RAM allows nodes with over-committed memory to swap pages over the network, storing them in the idle RAM of other nodes and avoiding swapping to slow, local disk. An adaptable network RAM system adjusts the amount of RAM currently available for storing remotely swapped pages in response to changes in nodespsila local RAM usage. It is important that network RAM systems provide reliability for remotely swapped page data. Without reliability, a single node failure can result in failure of unrelated processes running on other nodes by losing their remotely swapped pages. Adaptable network RAM systems pose extra difficulties in providing reliability because each nodepsilas capacity for storing remotely swapped pages changes over time, and because pages may move from node to node in response to these changes. Our novel dynamic RAID-based reliability solutions use idle RAM for storing page and reliability data, avoiding using slow disk for reliability. They are designed to work with the adaptive nature of our network RAM system (Nswap), allowing page and reliability data to migrate from node to node and allowing pages to be added to or removed from different parity groups. Additionally, page recovery runs concurrently with cluster applications, so that cluster applications do not have to wait until all data from a failed node is recovered before resuming execution. We present results comparing Nswap to disk swapping for a set of benchmarks running on our gigabit cluster. Our results show that reliable Nswap is up to 32 times faster than swapping to disk, and that there is virtually no impact on the performance of applications as they run concurrently with page recovery.","PeriodicalId":198768,"journal":{"name":"2008 IEEE International Conference on Cluster Computing","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115401184","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 17
Efficient one-copy MPI shared memory communication in Virtual Machines 虚拟机中高效的单拷贝MPI共享内存通信
Pub Date : 2008-10-31 DOI: 10.1109/CLUSTR.2008.4663761
Wei Huang, Matthew J. Koop, D. Panda
Efficient intra-node shared memory communication is important for high performance computing (HPC), especially with the emergence of multi-core architectures. As clusters continue to grow in size and complexity, the use of virtual machine (VM) technologies has been suggested to ease the increasing number of management issues. As demonstrated by earlier research, shared memory communication must be optimized for VMs to attain the native-level performance required by HPC centers. In this paper, we enhance intra-node shared memory communication for VM environments. We propose a one-copy approach. Instead of following the traditional approach used in most MPI implementations, copying data in and out of a pre-allocated shared memory region, our approach dynamically maps user buffers between VMs, allowing data to be directly copied to its destination. We also propose a grant/mapping cache to reduce expensive buffer mapping cost in VM environment. We integrate this approach into MVAPICH2, our implementation of MPI-2 library. For intra-node communication, we are able to reduce the large message latency in VM-based environments by up to 35%, and increase bandwidth by up to 38% even as compared with unmodified MVAPICH2 running in a native environment. Evaluation with the NAS Parallel Benchmarks suite shows up to 15% improvement.
高效的节点内共享内存通信对于高性能计算(HPC)非常重要,特别是随着多核架构的出现。随着集群的规模和复杂性不断增长,有人建议使用虚拟机(VM)技术来缓解越来越多的管理问题。正如先前的研究所表明的,共享内存通信必须针对vm进行优化,以达到HPC中心所需的本机级性能。在本文中,我们增强了虚拟机环境中的节点内共享内存通信。我们建议采用一份拷贝的方法。与大多数MPI实现中使用的传统方法(从预分配的共享内存区域复制数据)不同,我们的方法动态地映射虚拟机之间的用户缓冲区,允许数据直接复制到目的地。我们还提出了一个授权/映射缓存,以减少虚拟机环境中昂贵的缓冲区映射成本。我们将这种方法集成到MPI-2库的实现MVAPICH2中。对于节点内通信,我们能够将基于vm的环境中的大消息延迟减少多达35%,并且即使与在本机环境中运行的未修改的MVAPICH2相比,也可以将带宽增加多达38%。使用NAS Parallel benchmark套件进行的评估显示,改进幅度高达15%。
{"title":"Efficient one-copy MPI shared memory communication in Virtual Machines","authors":"Wei Huang, Matthew J. Koop, D. Panda","doi":"10.1109/CLUSTR.2008.4663761","DOIUrl":"https://doi.org/10.1109/CLUSTR.2008.4663761","url":null,"abstract":"Efficient intra-node shared memory communication is important for high performance computing (HPC), especially with the emergence of multi-core architectures. As clusters continue to grow in size and complexity, the use of virtual machine (VM) technologies has been suggested to ease the increasing number of management issues. As demonstrated by earlier research, shared memory communication must be optimized for VMs to attain the native-level performance required by HPC centers. In this paper, we enhance intra-node shared memory communication for VM environments. We propose a one-copy approach. Instead of following the traditional approach used in most MPI implementations, copying data in and out of a pre-allocated shared memory region, our approach dynamically maps user buffers between VMs, allowing data to be directly copied to its destination. We also propose a grant/mapping cache to reduce expensive buffer mapping cost in VM environment. We integrate this approach into MVAPICH2, our implementation of MPI-2 library. For intra-node communication, we are able to reduce the large message latency in VM-based environments by up to 35%, and increase bandwidth by up to 38% even as compared with unmodified MVAPICH2 running in a native environment. Evaluation with the NAS Parallel Benchmarks suite shows up to 15% improvement.","PeriodicalId":198768,"journal":{"name":"2008 IEEE International Conference on Cluster Computing","volume":"98 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123895157","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 17
A dependency-aware task-based programming environment for multi-core architectures 用于多核体系结构的依赖感知的基于任务的编程环境
Pub Date : 2008-10-31 DOI: 10.1109/CLUSTR.2008.4663765
Josep M. Pérez, Rosa M. Badia, Jesús Labarta
Parallel programming on SMP and multi-core architectures is hard. In this paper we present a programming model for those environments based on automatic function level parallelism that strives to be easy, flexible, portable, and performant. Its main trait is its ability to exploit task level parallelism by analyzing task dependencies at run time. We present the programming environment in the context of algorithms from several domains and pinpoint its benefits compared to other approaches. We discuss its execution model and its scheduler. Finally we analyze its performance and demonstrate that it offers reasonable performance without tuning, and that it can rival highly tuned libraries with minimal tuning effort.
在SMP和多核架构上并行编程是很困难的。在本文中,我们提出了一种基于自动功能级并行的编程模型,力求简单、灵活、可移植和高性能。它的主要特点是能够通过在运行时分析任务依赖关系来利用任务级并行性。我们在几个领域的算法背景下展示了编程环境,并指出了它与其他方法相比的好处。我们讨论了它的执行模型和调度程序。最后,我们分析了它的性能,并演示了它在没有调优的情况下提供了合理的性能,并且它可以用最少的调优工作与高度调优的库相媲美。
{"title":"A dependency-aware task-based programming environment for multi-core architectures","authors":"Josep M. Pérez, Rosa M. Badia, Jesús Labarta","doi":"10.1109/CLUSTR.2008.4663765","DOIUrl":"https://doi.org/10.1109/CLUSTR.2008.4663765","url":null,"abstract":"Parallel programming on SMP and multi-core architectures is hard. In this paper we present a programming model for those environments based on automatic function level parallelism that strives to be easy, flexible, portable, and performant. Its main trait is its ability to exploit task level parallelism by analyzing task dependencies at run time. We present the programming environment in the context of algorithms from several domains and pinpoint its benefits compared to other approaches. We discuss its execution model and its scheduler. Finally we analyze its performance and demonstrate that it offers reasonable performance without tuning, and that it can rival highly tuned libraries with minimal tuning effort.","PeriodicalId":198768,"journal":{"name":"2008 IEEE International Conference on Cluster Computing","volume":"90 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124855699","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 259
DLM: A distributed Large Memory System using remote memory swapping over cluster nodes DLM:在集群节点上使用远程内存交换的分布式大内存系统
Pub Date : 2008-10-31 DOI: 10.1109/CLUSTR.2008.4663780
H. Midorikawa, M. Kurokawa, R. Himeno, M. Sato
Emerging 64 bitOSpsilas supply a huge amount of memory address space that is essential for new applications using very large data. It is expected that the memory in connected nodes can be used to store swapped pages efficiently, especially in a dedicated cluster which has a high-speed network such as 10 GbE and Infiniband. In this paper, we propose the distributed large memory system (DLM), which provides very large virtual memory by using remote memory distributed over the nodes in a cluster. The performance of DLM programs using remote memory is compared to ordinary programs using local memory. The results of STREAM, NPB and Himeno benchmarks show that the DLM achieves better performance than other remote paging schemes using a block swap device to access remote memory. In addition to performance, DLM offers the advantages of easy availability and high portability, because it is a user-level software without the need for special hardware. To obtain high performance, the DLM can tune its parameters independently from kernel swap parameters. We also found that DLMpsilas independence of kernel swapping provides more stable behavior.
新兴的64位ospsilas提供了大量的内存地址空间,这对于使用非常大的数据的新应用程序是必不可少的。预计连接节点中的内存可以用于有效地存储交换页面,特别是在具有高速网络(如10gbe和Infiniband)的专用集群中。本文提出了分布式大内存系统(distributed large memory system, DLM),它通过使用分布在集群节点上的远程内存来提供非常大的虚拟内存。将使用远程内存的DLM程序的性能与使用本地内存的普通程序进行比较。STREAM、NPB和Himeno基准测试的结果表明,DLM比使用块交换设备访问远程内存的其他远程分页方案实现了更好的性能。除了性能之外,DLM还提供了易于获得和高可移植性的优点,因为它是一种用户级软件,不需要特殊的硬件。为了获得高性能,DLM可以独立于内核交换参数调优其参数。我们还发现,内核交换的DLMpsilas独立性提供了更稳定的行为。
{"title":"DLM: A distributed Large Memory System using remote memory swapping over cluster nodes","authors":"H. Midorikawa, M. Kurokawa, R. Himeno, M. Sato","doi":"10.1109/CLUSTR.2008.4663780","DOIUrl":"https://doi.org/10.1109/CLUSTR.2008.4663780","url":null,"abstract":"Emerging 64 bitOSpsilas supply a huge amount of memory address space that is essential for new applications using very large data. It is expected that the memory in connected nodes can be used to store swapped pages efficiently, especially in a dedicated cluster which has a high-speed network such as 10 GbE and Infiniband. In this paper, we propose the distributed large memory system (DLM), which provides very large virtual memory by using remote memory distributed over the nodes in a cluster. The performance of DLM programs using remote memory is compared to ordinary programs using local memory. The results of STREAM, NPB and Himeno benchmarks show that the DLM achieves better performance than other remote paging schemes using a block swap device to access remote memory. In addition to performance, DLM offers the advantages of easy availability and high portability, because it is a user-level software without the need for special hardware. To obtain high performance, the DLM can tune its parameters independently from kernel swap parameters. We also found that DLMpsilas independence of kernel swapping provides more stable behavior.","PeriodicalId":198768,"journal":{"name":"2008 IEEE International Conference on Cluster Computing","volume":"114 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132565255","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 24
Enabling lock-free concurrent fine-grain access to massive distributed data: Application to supernovae detection 实现对海量分布式数据的无锁并发细粒度访问:超新星探测中的应用
Pub Date : 2008-10-13 DOI: 10.1109/CLUSTR.2008.4663787
Bogdan Nicolae, Gabriel Antoniu, L. Bougé
We consider the problem of efficiently managing massive data in a large-scale distributed environment. We consider data strings of size in the order of Terabytes, shared and accessed by concurrent clients. On each individual access, a segment of a string, of the order of Megabytes, is read or modified. Our goal is to provide the clients with efficient fine-grain access the data string as concurrently as possible, without locking the string itself. This issue is crucial in the context of applications in the field of astronomy, databases, data mining and multimedia. We illustrate these requirements with the case of an application for searching supernovae. Our solution relies on distributed, RAM-based data storage, while leveraging a DHT-based, parallel metadata management scheme. The proposed architecture and algorithms have been validated through a software prototype and evaluated in a cluster environment.
我们考虑在大规模分布式环境中有效管理海量数据的问题。我们认为数据字符串的大小按tb的顺序,由并发客户端共享和访问。在每次单独访问时,读取或修改一个以兆字节为数量级的字符串段。我们的目标是在不锁定字符串本身的情况下,尽可能并发地为客户端提供高效的细粒度访问数据字符串。这个问题在天文学、数据库、数据挖掘和多媒体领域的应用中是至关重要的。我们以搜索超新星的应用程序为例说明这些要求。我们的解决方案依赖于分布式的、基于ram的数据存储,同时利用基于dht的并行元数据管理方案。提出的架构和算法已经通过软件原型验证,并在集群环境中进行了评估。
{"title":"Enabling lock-free concurrent fine-grain access to massive distributed data: Application to supernovae detection","authors":"Bogdan Nicolae, Gabriel Antoniu, L. Bougé","doi":"10.1109/CLUSTR.2008.4663787","DOIUrl":"https://doi.org/10.1109/CLUSTR.2008.4663787","url":null,"abstract":"We consider the problem of efficiently managing massive data in a large-scale distributed environment. We consider data strings of size in the order of Terabytes, shared and accessed by concurrent clients. On each individual access, a segment of a string, of the order of Megabytes, is read or modified. Our goal is to provide the clients with efficient fine-grain access the data string as concurrently as possible, without locking the string itself. This issue is crucial in the context of applications in the field of astronomy, databases, data mining and multimedia. We illustrate these requirements with the case of an application for searching supernovae. Our solution relies on distributed, RAM-based data storage, while leveraging a DHT-based, parallel metadata management scheme. The proposed architecture and algorithms have been validated through a software prototype and evaluated in a cluster environment.","PeriodicalId":198768,"journal":{"name":"2008 IEEE International Conference on Cluster Computing","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125304478","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
Live and incremental whole-system migration of virtual machines using block-bitmap 使用块位图实现虚拟机的实时和增量全系统迁移
Pub Date : 2008-09-01 DOI: 10.1109/CLUSTR.2008.4663760
Yingwei Luo, Binbin Zhang, Xiaolin Wang, Zhenlin Wang, Yifeng Sun, Haogang Chen
In this paper, we describe a whole-system live migration scheme, which transfers the whole system run-time state, including CPU state, memory data, and local disk storage, of the virtual machine (VM). To minimize the downtime caused by migrating large disk storage data and keep data integrity and consistency, we propose a three-phase migration (TPM) algorithm. To facilitate the migration back to initial source machine, we use an incremental migration (IM) algorithm to reduce the amount of the data to be migrated. Block-bitmap is used to track all the write accesses to the local disk storage during the migration. Synchronization of the local disk storage in the migration is performed according to the block-bitmap. Experiments show that our algorithms work well even when I/O-intensive workloads are running in the migrated VM. The downtime of the migration is around 100 milliseconds, close to shared-storage migration. Total migration time is greatly reduced using IM. The block-bitmap based synchronization mechanism is simple and effective. Performance overhead of recording all the writes on migrated VM is very low.
在本文中,我们描述了一种全系统实时迁移方案,该方案传输虚拟机(VM)的整个系统运行时状态,包括CPU状态、内存数据和本地磁盘存储。为了最大限度地减少迁移大磁盘存储数据所造成的停机时间,并保持数据的完整性和一致性,我们提出了一种三相迁移(TPM)算法。为了方便迁移回初始源机器,我们使用增量迁移(IM)算法来减少要迁移的数据量。块位图用于跟踪迁移过程中对本地磁盘存储的所有写访问。迁移过程中本地磁盘存储按照块位图同步。实验表明,即使在迁移的VM中运行I/ o密集型工作负载时,我们的算法也能很好地工作。迁移的停机时间约为100毫秒,接近于共享存储迁移。使用IM大大减少了总迁移时间。基于块位图的同步机制简单有效。在迁移后的虚拟机上记录所有写操作的性能开销非常低。
{"title":"Live and incremental whole-system migration of virtual machines using block-bitmap","authors":"Yingwei Luo, Binbin Zhang, Xiaolin Wang, Zhenlin Wang, Yifeng Sun, Haogang Chen","doi":"10.1109/CLUSTR.2008.4663760","DOIUrl":"https://doi.org/10.1109/CLUSTR.2008.4663760","url":null,"abstract":"In this paper, we describe a whole-system live migration scheme, which transfers the whole system run-time state, including CPU state, memory data, and local disk storage, of the virtual machine (VM). To minimize the downtime caused by migrating large disk storage data and keep data integrity and consistency, we propose a three-phase migration (TPM) algorithm. To facilitate the migration back to initial source machine, we use an incremental migration (IM) algorithm to reduce the amount of the data to be migrated. Block-bitmap is used to track all the write accesses to the local disk storage during the migration. Synchronization of the local disk storage in the migration is performed according to the block-bitmap. Experiments show that our algorithms work well even when I/O-intensive workloads are running in the migrated VM. The downtime of the migration is around 100 milliseconds, close to shared-storage migration. Total migration time is greatly reduced using IM. The block-bitmap based synchronization mechanism is simple and effective. Performance overhead of recording all the writes on migrated VM is very low.","PeriodicalId":198768,"journal":{"name":"2008 IEEE International Conference on Cluster Computing","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114804069","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 116
Magnet: A novel scheduling policy for power reduction in cluster with virtual machines Magnet:一种用于虚拟机集群中降低功耗的新调度策略
Pub Date : 2008-09-01 DOI: 10.1109/CLUSTR.2008.4663751
Liting Hu, Hai Jin, Xiaofei Liao, Xianjie Xiong, Haikun Liu
The concept of green computing has attracted much attention recently in cluster computing. However, previous local approaches focused on saving the energy cost of the components in a single workstation without a global vision on the whole cluster, so it achieved undesirable power reduction effect. Other cluster-wide energy saving techniques could only be applied to homogeneous workstations and specific applications. This paper describes the design and implementation of a novel approach that uses live migration of virtual machines to transfer load among the nodes on a multilayer ring-based overlay. This scheme can reduce the power consumption greatly by regarding all the cluster nodes as a whole. Plus, it can be applied to both the homogeneous and heterogeneous servers. Experimental measurements show that the new method can reduce the power consumption by 74.8% over base at most with certain adjustably acceptable overhead. The effectiveness and performance insights are also analytically verified.
近年来,绿色计算的概念在集群计算领域受到了广泛的关注。然而,以往的局部方法侧重于节省单个工作站中组件的能源成本,而没有考虑到整个集群的全局,因此无法达到理想的降电效果。其他集群范围内的节能技术只能应用于同类工作站和特定应用程序。本文描述了一种新方法的设计和实现,该方法使用虚拟机的实时迁移在多层环覆盖层的节点之间传输负载。该方案将集群中的所有节点视为一个整体,大大降低了功耗。此外,它既可以应用于同构服务器,也可以应用于异构服务器。实验结果表明,在一定的可接受开销下,该方法最多可将功耗降低74.8%。对有效性和性能的见解也进行了分析验证。
{"title":"Magnet: A novel scheduling policy for power reduction in cluster with virtual machines","authors":"Liting Hu, Hai Jin, Xiaofei Liao, Xianjie Xiong, Haikun Liu","doi":"10.1109/CLUSTR.2008.4663751","DOIUrl":"https://doi.org/10.1109/CLUSTR.2008.4663751","url":null,"abstract":"The concept of green computing has attracted much attention recently in cluster computing. However, previous local approaches focused on saving the energy cost of the components in a single workstation without a global vision on the whole cluster, so it achieved undesirable power reduction effect. Other cluster-wide energy saving techniques could only be applied to homogeneous workstations and specific applications. This paper describes the design and implementation of a novel approach that uses live migration of virtual machines to transfer load among the nodes on a multilayer ring-based overlay. This scheme can reduce the power consumption greatly by regarding all the cluster nodes as a whole. Plus, it can be applied to both the homogeneous and heterogeneous servers. Experimental measurements show that the new method can reduce the power consumption by 74.8% over base at most with certain adjustably acceptable overhead. The effectiveness and performance insights are also analytically verified.","PeriodicalId":198768,"journal":{"name":"2008 IEEE International Conference on Cluster Computing","volume":"152 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122861785","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 84
A multicore-enabled multirail communication engine 支持多核的多轨通信引擎
Pub Date : 2008-09-01 DOI: 10.1109/CLUSTR.2008.4663788
E. Brunet, François Trahay, Alexandre Denis
The current trend in clusters architecture leads toward a massive use of multicore chips. This hardware evolution raises bottleneck issues at the network interface level. The use of multiple parallel networks allows to overcome this problem as it provides an higher aggregate bandwidth. But this bandwidth remains theoretical as only a few communication libraries are able to exploit multiple networks. In this paper, we present an optimization strategy for the NEWMADELEINE communication library. This strategy is able to efficiently exploit parallel interconnect links. By sampling each networkpsilas capabilities, it is possible to estimate a transfer duration a priori. Splitting messages and sending chunks of messages over parallel links can thus be performed efficiently to reach the theoretical aggregate bandwidth. NEWMADELEINE is multithreaded and exploits multicore chips to send small packets, that involve CPU-consuming copies, in parallel.
当前集群架构的趋势是大量使用多核芯片。这种硬件演变在网络接口级别引发了瓶颈问题。使用多个并行网络可以克服这个问题,因为它提供了更高的聚合带宽。但是这个带宽仍然是理论上的,因为只有少数通信库能够利用多个网络。本文提出了一种针对NEWMADELEINE通信库的优化策略。该策略能够有效地利用并行互连链路。通过对每个网络的能力进行抽样,可以先验地估计传输持续时间。因此,拆分消息和在并行链路上发送消息块可以有效地执行,以达到理论聚合带宽。NEWMADELEINE是多线程的,利用多核芯片发送小数据包,这些数据包涉及cpu消耗副本,并行。
{"title":"A multicore-enabled multirail communication engine","authors":"E. Brunet, François Trahay, Alexandre Denis","doi":"10.1109/CLUSTR.2008.4663788","DOIUrl":"https://doi.org/10.1109/CLUSTR.2008.4663788","url":null,"abstract":"The current trend in clusters architecture leads toward a massive use of multicore chips. This hardware evolution raises bottleneck issues at the network interface level. The use of multiple parallel networks allows to overcome this problem as it provides an higher aggregate bandwidth. But this bandwidth remains theoretical as only a few communication libraries are able to exploit multiple networks. In this paper, we present an optimization strategy for the NEWMADELEINE communication library. This strategy is able to efficiently exploit parallel interconnect links. By sampling each networkpsilas capabilities, it is possible to estimate a transfer duration a priori. Splitting messages and sending chunks of messages over parallel links can thus be performed efficiently to reach the theoretical aggregate bandwidth. NEWMADELEINE is multithreaded and exploits multicore chips to send small packets, that involve CPU-consuming copies, in parallel.","PeriodicalId":198768,"journal":{"name":"2008 IEEE International Conference on Cluster Computing","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126766888","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Intelligent compilers 智能的编译器
Pub Date : 2008-09-01 DOI: 10.1109/CLUSTR.2008.4663796
John Cavazos
The industry is now in agreement that the future of architecture design lies in multiple cores. As a consequence, all computer systems today, from embedded devices to petascale computing systems, are being developed using multicore processors. Although researchers in industry and academia are exploring many different multicore hardware design choices, most agree that developing portable software that achieves high performance on multicore processors is a major unsolved problem. We now see a plethora of architectural features, with little consensus on how the computation, memory, and communication structures in multicore systems will be organized. The wide disparity in hardware systems available has made it nearly impossible to write code that is portable in functionality while still taking advantage of the performance potential of each system. In this paper, we propose exploring the viability of developing intelligent compilers, focusing on key components that will allow application portability while still achieving high performance.
业界现在一致认为,架构设计的未来在于多核。因此,今天所有的计算机系统,从嵌入式设备到千兆级计算系统,都是使用多核处理器开发的。尽管工业界和学术界的研究人员正在探索许多不同的多核硬件设计选择,但大多数人都认为,开发在多核处理器上实现高性能的可移植软件是一个主要的未解决问题。我们现在看到了大量的架构特性,但对于多核系统中的计算、内存和通信结构如何组织却没有达成共识。可用硬件系统的巨大差异使得几乎不可能编写功能可移植的代码,同时仍然利用每个系统的性能潜力。在本文中,我们建议探索开发智能编译器的可行性,重点关注将允许应用程序可移植性同时仍然实现高性能的关键组件。
{"title":"Intelligent compilers","authors":"John Cavazos","doi":"10.1109/CLUSTR.2008.4663796","DOIUrl":"https://doi.org/10.1109/CLUSTR.2008.4663796","url":null,"abstract":"The industry is now in agreement that the future of architecture design lies in multiple cores. As a consequence, all computer systems today, from embedded devices to petascale computing systems, are being developed using multicore processors. Although researchers in industry and academia are exploring many different multicore hardware design choices, most agree that developing portable software that achieves high performance on multicore processors is a major unsolved problem. We now see a plethora of architectural features, with little consensus on how the computation, memory, and communication structures in multicore systems will be organized. The wide disparity in hardware systems available has made it nearly impossible to write code that is portable in functionality while still taking advantage of the performance potential of each system. In this paper, we propose exploring the viability of developing intelligent compilers, focusing on key components that will allow application portability while still achieving high performance.","PeriodicalId":198768,"journal":{"name":"2008 IEEE International Conference on Cluster Computing","volume":"81 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2008-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132346524","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 11
期刊
2008 IEEE International Conference on Cluster Computing
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1