首页 > 最新文献

2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid最新文献

英文 中文
Dynamic Provisioning of Virtual Organization Clusters 动态发放虚拟组织集群
Pub Date : 2009-05-18 DOI: 10.1109/CCGRID.2009.37
M. Murphy, Brandon Kagey, M. Fenn, S. Goasguen
Virtual Organization Clusters are systems comprised of virtual machines that provide dedicated computing clusters for each individual Virtual Organization. The design of these clusters allows individual virtual machines to be independent of the underlying physical hardware, potentially allowing virtual clusters to span multiple grid sites. A major challenge in using Virtual Organization Clusters as a grid computing abstraction arises from the need to schedule and provision physical resources to run the virtual machines.This paper describes a virtual cluster scheduler implementation based on the Condor High Throughput Computing system. By means of real-time monitoring of the Condor job queue, virtual machines that belong to individual Virtual Organizations are provisioned and booted. Jobs belonging to each Virtual Organization are then run on the organization-specific virtual machines, which form a cluster dedicated to the specific organization. Once the queued jobs have executed, the virtual machines are terminated, thereby allowing the physical resources to be re-claimed. Tests of this system were conducted using synthetic workloads, demonstrating that dynamic provisioning of virtual machines preserves system throughput for all but the shortest-running of grid jobs, without undue increase in scheduling latency.
虚拟组织集群是由虚拟机组成的系统,为每个单独的虚拟组织提供专用的计算集群。这些集群的设计允许单个虚拟机独立于底层物理硬件,从而潜在地允许虚拟集群跨越多个网格站点。使用Virtual Organization Clusters作为网格计算抽象的一个主要挑战来自于需要调度和提供物理资源来运行虚拟机。本文介绍了一种基于Condor高吞吐量计算系统的虚拟集群调度程序的实现。通过实时监视Condor作业队列,可以供应和引导属于各个虚拟组织的虚拟机。然后,属于每个虚拟组织的作业在特定于组织的虚拟机上运行,这些虚拟机形成了专用于特定组织的集群。一旦队列作业执行完毕,虚拟机就会被终止,从而允许重新占用物理资源。使用合成工作负载对该系统进行了测试,结果表明,除了运行时间最短的网格作业之外,虚拟机的动态配置可以保持所有网格作业的系统吞吐量,而不会过度增加调度延迟。
{"title":"Dynamic Provisioning of Virtual Organization Clusters","authors":"M. Murphy, Brandon Kagey, M. Fenn, S. Goasguen","doi":"10.1109/CCGRID.2009.37","DOIUrl":"https://doi.org/10.1109/CCGRID.2009.37","url":null,"abstract":"Virtual Organization Clusters are systems comprised of virtual machines that provide dedicated computing clusters for each individual Virtual Organization. The design of these clusters allows individual virtual machines to be independent of the underlying physical hardware, potentially allowing virtual clusters to span multiple grid sites. A major challenge in using Virtual Organization Clusters as a grid computing abstraction arises from the need to schedule and provision physical resources to run the virtual machines.This paper describes a virtual cluster scheduler implementation based on the Condor High Throughput Computing system. By means of real-time monitoring of the Condor job queue, virtual machines that belong to individual Virtual Organizations are provisioned and booted. Jobs belonging to each Virtual Organization are then run on the organization-specific virtual machines, which form a cluster dedicated to the specific organization. Once the queued jobs have executed, the virtual machines are terminated, thereby allowing the physical resources to be re-claimed. Tests of this system were conducted using synthetic workloads, demonstrating that dynamic provisioning of virtual machines preserves system throughput for all but the shortest-running of grid jobs, without undue increase in scheduling latency.","PeriodicalId":118263,"journal":{"name":"2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid","volume":"144 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116433315","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 67
Web-Based Visualization of Atmospheric Nucleation Processes Using Java3D 使用Java3D的基于web的大气成核过程可视化
Pub Date : 2009-05-18 DOI: 10.1109/CCGRID.2009.56
Rui Ding, Jinzhu Gao, Bin Chen, J. Siepmann, Yi Liu
With the development of science, research on data analysis is becoming increasingly important. However, sometimes it is difficult to draw a conclusion from a complex data set. Data Visualization has been widely used for people to understand more about their datasets by representing the data in a way that visually highlights the relationships. As the size of data grows exponentially, keeping multiple local copies of the data becomes unrealistic for a collaborative research project. In this paper, we design and develop a cybertool, CT-IANP, which supports collaborative research in the area of atmospheric nucleation. The paper shows how Java 3D, web-based tools, and other techniques are used to achieve the goal.
随着科学的发展,对数据分析的研究变得越来越重要。然而,有时很难从复杂的数据集中得出结论。数据可视化已经被广泛应用,人们可以通过一种直观地突出关系的方式来表示数据,从而更好地了解他们的数据集。随着数据规模呈指数级增长,对于合作研究项目来说,保留多个数据本地副本变得不现实。在本文中,我们设计和开发了一个网络工具,CT-IANP,它支持在大气成核领域的合作研究。本文展示了如何使用Java 3D、基于web的工具和其他技术来实现这一目标。
{"title":"Web-Based Visualization of Atmospheric Nucleation Processes Using Java3D","authors":"Rui Ding, Jinzhu Gao, Bin Chen, J. Siepmann, Yi Liu","doi":"10.1109/CCGRID.2009.56","DOIUrl":"https://doi.org/10.1109/CCGRID.2009.56","url":null,"abstract":"With the development of science, research on data analysis is becoming increasingly important. However, sometimes it is difficult to draw a conclusion from a complex data set. Data Visualization has been widely used for people to understand more about their datasets by representing the data in a way that visually highlights the relationships. As the size of data grows exponentially, keeping multiple local copies of the data becomes unrealistic for a collaborative research project. In this paper, we design and develop a cybertool, CT-IANP, which supports collaborative research in the area of atmospheric nucleation. The paper shows how Java 3D, web-based tools, and other techniques are used to achieve the goal.","PeriodicalId":118263,"journal":{"name":"2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131661957","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
BLAST Application with Data-Aware Desktop Grid Middleware 具有数据感知桌面网格中间件的BLAST应用程序
Pub Date : 2009-05-18 DOI: 10.1109/CCGRID.2009.91
Haiwu He, G. Fedak, B. Tang, F. Cappello
There exists numerous Grid middleware to develop and execute programs on the computational Grid, but they still require intensive work from their users. BitDew is made to facilitate the usage of large scale Grid with dynamic, heterogeneous, volatile and highly distributed computing resources for applications that require a huge amount of data processing. Data-intensive applications form an important class of applications for the e-Science community which require secure and coordinated access to large datasets, wide-area transfers and broad distribution ofTeraBytes of data while keeping track of multiple data replicas. In genetic biology, gene sequences comparison and analysis are the most basic routines. With the considerable increase of sequences to analyze, we need more and more computing power as well as efficient solution to manage data. In this work, we investigate the advantages of using a new Desktop Grid middleware BitDew, designed for large scale data management.Our contribution is two-fold: firstly, we introduce a data-driven Master/Slave programming model and we present an implementation of BLAST over BitDew following this model, secondly, we present extensive experimental and simulation results which demonstrate the effectiveness and scalability of our approach. We evaluate the benefit of multi-protocol data distribution to achieve remarkable speedups, we report on the ability to cope with highly volatile environment with relative performance degradation, we show the benefit of data replication in Grid with heterogeneous resource performance and we evaluate the combination of data fault tolerance and data replication when computing on volatileresources.
有许多网格中间件可以在计算网格上开发和执行程序,但是它们仍然需要用户进行大量的工作。BitDew旨在为需要大量数据处理的应用程序提供具有动态、异构、易失性和高度分布式计算资源的大规模网格。数据密集型应用构成了电子科学社区的重要应用类别,它需要安全协调地访问大型数据集、广域传输和tb级数据的广泛分发,同时保持对多个数据副本的跟踪。在遗传生物学中,基因序列比较和分析是最基本的程序。随着需要分析的序列的大量增加,我们需要越来越强的计算能力以及高效的数据管理解决方案。在这项工作中,我们研究了使用新的桌面网格中间件BitDew的优势,BitDew是为大规模数据管理而设计的。我们的贡献是双重的:首先,我们引入了一个数据驱动的主/从编程模型,并根据该模型在BitDew上提出了BLAST的实现,其次,我们提出了广泛的实验和仿真结果,证明了我们方法的有效性和可扩展性。我们评估了多协议数据分布实现显著提速的好处,我们报告了处理相对性能下降的高度易变环境的能力,我们展示了网格中具有异构资源性能的数据复制的好处,我们评估了在易变资源上计算时数据容错和数据复制的组合。
{"title":"BLAST Application with Data-Aware Desktop Grid Middleware","authors":"Haiwu He, G. Fedak, B. Tang, F. Cappello","doi":"10.1109/CCGRID.2009.91","DOIUrl":"https://doi.org/10.1109/CCGRID.2009.91","url":null,"abstract":"There exists numerous Grid middleware to develop and execute programs on the computational Grid, but they still require intensive work from their users. BitDew is made to facilitate the usage of large scale Grid with dynamic, heterogeneous, volatile and highly distributed computing resources for applications that require a huge amount of data processing. Data-intensive applications form an important class of applications for the e-Science community which require secure and coordinated access to large datasets, wide-area transfers and broad distribution ofTeraBytes of data while keeping track of multiple data replicas. In genetic biology, gene sequences comparison and analysis are the most basic routines. With the considerable increase of sequences to analyze, we need more and more computing power as well as efficient solution to manage data. In this work, we investigate the advantages of using a new Desktop Grid middleware BitDew, designed for large scale data management.Our contribution is two-fold: firstly, we introduce a data-driven Master/Slave programming model and we present an implementation of BLAST over BitDew following this model, secondly, we present extensive experimental and simulation results which demonstrate the effectiveness and scalability of our approach. We evaluate the benefit of multi-protocol data distribution to achieve remarkable speedups, we report on the ability to cope with highly volatile environment with relative performance degradation, we show the benefit of data replication in Grid with heterogeneous resource performance and we evaluate the combination of data fault tolerance and data replication when computing on volatileresources.","PeriodicalId":118263,"journal":{"name":"2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid","volume":"17 4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130661717","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 18
Hybrid Re-scheduling Mechanisms for Workflow Applications on Multi-cluster Grid 多集群网格下工作流应用的混合重调度机制
Pub Date : 2009-05-18 DOI: 10.1109/CCGRID.2009.60
Yang Zhang, C. Koelbel, K. Cooper
Grid computing is now a viable computational paradigm for executing large scale workflow applications. However, many aspects of performance optimization remain challenging. In this paper, we focus on the workflow scheduling mechanism. While there is much work on static scheduling approaches for workflow applications in parallel environments, little work has been done on a real-world multi-cluster Grid environment. Since a typical Grid environment is dynamic,  we propose a new cluster-based scheduling mechanism that dynamically executes a top-down static scheduling algorithm using the real-time feedback from the execution monitor. We also propose a novel two phase migration mechanism that mitigates the effect of a possible bad reschedule decision. Our experimental results show that this approach achieves the best performance among all the scheduling approaches we implemented on both reserved resources and those with external loads.
网格计算现在是执行大规模工作流应用程序的可行计算范式。然而,性能优化的许多方面仍然具有挑战性。本文主要研究工作流调度机制。虽然在并行环境中工作流应用程序的静态调度方法方面有很多工作,但在实际的多集群网格环境中做的工作很少。由于典型的网格环境是动态的,我们提出了一种新的基于集群的调度机制,该机制利用执行监视器的实时反馈动态执行自顶向下的静态调度算法。我们还提出了一种新的两阶段迁移机制,以减轻可能出现的不良重新调度决策的影响。实验结果表明,在所有的调度方法中,该方法在预留资源和有外部负载的调度方法中都取得了最好的性能。
{"title":"Hybrid Re-scheduling Mechanisms for Workflow Applications on Multi-cluster Grid","authors":"Yang Zhang, C. Koelbel, K. Cooper","doi":"10.1109/CCGRID.2009.60","DOIUrl":"https://doi.org/10.1109/CCGRID.2009.60","url":null,"abstract":"Grid computing is now a viable computational paradigm for executing large scale workflow applications. However, many aspects of performance optimization remain challenging. In this paper, we focus on the workflow scheduling mechanism. While there is much work on static scheduling approaches for workflow applications in parallel environments, little work has been done on a real-world multi-cluster Grid environment. Since a typical Grid environment is dynamic,  we propose a new cluster-based scheduling mechanism that dynamically executes a top-down static scheduling algorithm using the real-time feedback from the execution monitor. We also propose a novel two phase migration mechanism that mitigates the effect of a possible bad reschedule decision. Our experimental results show that this approach achieves the best performance among all the scheduling approaches we implemented on both reserved resources and those with external loads.","PeriodicalId":118263,"journal":{"name":"2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132733186","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 29
Utility Driven Adaptive Work?ow Execution 效用驱动的适应性工作?噢执行
Pub Date : 2009-05-18 DOI: 10.1109/CCGRID.2009.15
Kevin Lee, N. Paton, R. Sakellariou, A. Fernandes
Workflows are widely used in applications that require coordinated use of computational resources. Workflow definition languages typically abstract over some aspects of the way in which a workflow is to be executed, such as the level of parallelism to be used or the physical resources to be deployed. As a result, a workflow management system has responsibility for establishing how best to map tasks within a workflow to the available resources. As workflows are typically run over shared resources, and thus face unpredictable and changing resource capabilties, there may be benefit to be derived from adapting the task-to-resource mapping while a workflow is executing.  This paper describes the use of utility functions to express the relative merits of alternative mappings; in essence, a utility function can be used to give a score to a candidate mapping, and the exploration of alternative mappings can be cast as an optimization problem. In this approach, changing the utility function allows adaptations to be carried out with a view to meeting different objectives.  The contributions of this paper include: (i) a description of how adaptive workflow execution can be expressed as an optimization problem where the objective of the adaptation is to maximize some property expressed as a utility function; (ii) a description of how the approach has been applied to support adaptive workflow execution in grids; and (iii) an experimental evaluation of the resulting approach for alternative utility measures based on response time and profit.
工作流广泛应用于需要协调使用计算资源的应用程序中。工作流定义语言通常对工作流执行方式的某些方面进行抽象,例如要使用的并行性级别或要部署的物理资源。因此,工作流管理系统有责任确定如何最好地将工作流中的任务映射到可用资源。由于工作流通常在共享资源上运行,因此面临不可预测和不断变化的资源功能,因此在执行工作流时调整任务到资源的映射可能会带来好处。本文描述了使用效用函数来表示备选映射的相对优点;从本质上讲,可以使用效用函数为候选映射打分,而对备选映射的探索可以作为一个优化问题。在这种方法中,更改实用程序功能允许进行调整,以满足不同的目标。本文的贡献包括:(i)描述了如何将自适应工作流的执行表示为一个优化问题,其中自适应的目标是最大化表示为效用函数的某些属性;(ii)描述如何应用该方法来支持网格中的自适应工作流执行;(iii)对基于响应时间和利润的替代效用措施的最终方法进行实验评估。
{"title":"Utility Driven Adaptive Work?ow Execution","authors":"Kevin Lee, N. Paton, R. Sakellariou, A. Fernandes","doi":"10.1109/CCGRID.2009.15","DOIUrl":"https://doi.org/10.1109/CCGRID.2009.15","url":null,"abstract":"Workflows are widely used in applications that require coordinated use of computational resources. Workflow definition languages typically abstract over some aspects of the way in which a workflow is to be executed, such as the level of parallelism to be used or the physical resources to be deployed. As a result, a workflow management system has responsibility for establishing how best to map tasks within a workflow to the available resources. As workflows are typically run over shared resources, and thus face unpredictable and changing resource capabilties, there may be benefit to be derived from adapting the task-to-resource mapping while a workflow is executing.  This paper describes the use of utility functions to express the relative merits of alternative mappings; in essence, a utility function can be used to give a score to a candidate mapping, and the exploration of alternative mappings can be cast as an optimization problem. In this approach, changing the utility function allows adaptations to be carried out with a view to meeting different objectives.  The contributions of this paper include: (i) a description of how adaptive workflow execution can be expressed as an optimization problem where the objective of the adaptation is to maximize some property expressed as a utility function; (ii) a description of how the approach has been applied to support adaptive workflow execution in grids; and (iii) an experimental evaluation of the resulting approach for alternative utility measures based on response time and profit.","PeriodicalId":118263,"journal":{"name":"2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133582934","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 25
Towards Visualization Scalability through Time Intervals and Hierarchical Organization of Monitoring Data 通过时间间隔和分层组织实现监控数据的可视化可扩展性
Pub Date : 2009-05-18 DOI: 10.1109/CCGRID.2009.19
L. Schnorr, Guillaume Huard, P. Navaux
Highly distributed systems such as Grids are used today to the execution of large-scale parallel applications. The behavior analysis of these applications is not trivial. The complexity appears because of the event correlation among processes, external influences like time-sharing mechanisms and saturation of network links, and also the amount of data that registers the application behavior. Almost all visualization tools to analysis of parallel applications offer a space-time representation of the application behavior. This paper presents a novel technique that combines traces from grid applications with a treemap visualization of the data. With this combination, we dynamically create an annotated hierarchical structure that represents the application behavior for the selected time interval. The experiments in the grid show that we can readily use our technique to the analysis of large-scale parallel applications with thousands of processes.
像网格这样的高度分布式系统现在被用于执行大规模并行应用程序。这些应用程序的行为分析并不简单。复杂性的出现是由于进程之间的事件相关性、外部影响(如分时机制和网络链接的饱和)以及记录应用程序行为的数据量。几乎所有用于分析并行应用程序的可视化工具都提供了应用程序行为的时空表示。本文提出了一种新的技术,将来自网格应用程序的轨迹与数据的树图可视化相结合。通过这种组合,我们可以动态地创建一个带注释的层次结构,该结构表示所选时间间隔的应用程序行为。在网格中的实验表明,我们可以很容易地将我们的技术应用于具有数千个进程的大规模并行应用程序的分析。
{"title":"Towards Visualization Scalability through Time Intervals and Hierarchical Organization of Monitoring Data","authors":"L. Schnorr, Guillaume Huard, P. Navaux","doi":"10.1109/CCGRID.2009.19","DOIUrl":"https://doi.org/10.1109/CCGRID.2009.19","url":null,"abstract":"Highly distributed systems such as Grids are used today to the execution of large-scale parallel applications. The behavior analysis of these applications is not trivial. The complexity appears because of the event correlation among processes, external influences like time-sharing mechanisms and saturation of network links, and also the amount of data that registers the application behavior. Almost all visualization tools to analysis of parallel applications offer a space-time representation of the application behavior. This paper presents a novel technique that combines traces from grid applications with a treemap visualization of the data. With this combination, we dynamically create an annotated hierarchical structure that represents the application behavior for the selected time interval. The experiments in the grid show that we can readily use our technique to the analysis of large-scale parallel applications with thousands of processes.","PeriodicalId":118263,"journal":{"name":"2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114653717","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 14
Adaptive Resource Indexing Technique for Unstructured Peer-to-Peer Networks 非结构化点对点网络的自适应资源索引技术
Pub Date : 2009-05-18 DOI: 10.1109/CCGRID.2009.41
S. Lerthirunwong, N. Maruyama, S. Matsuoka
Searching for particular resources in a large-scale decentralized unstructured network can be very difficult since there is no centralized management to provide the specific location of resources. Moreover, the dynamic behavior of networks and the diversity of user behavior cause the search more complex and may not guarantee success. To address the problems, we propose a new adaptive resource indexing technique that aims to increase both efficiency and quality of the search by reducing both messages and time required for each query. Our approach consists of two complementary techniques. One is an index selection technique that selectively keeps the indices at each peer to increase the chance of successful queries with minimum space requirement. Another is an index distribution technique that automatically adjusts index distribution rate based on the search performance to optimize both the search performance and overhead. We simulate the technique in various network conditions and the results show that our technique is effective in decreasing hop counts and messages needed for resolving queries with only small overhead. It decreases the average hop count by up to 44% with 75%-less messages when used with flooding based queries even facing high churn. Furthermore, the query success rate with a limited timeout condition also increases, approaching nearly to 100%.
在大规模分散的非结构化网络中搜索特定资源可能非常困难,因为没有集中管理来提供资源的具体位置。此外,网络的动态行为和用户行为的多样性使搜索更加复杂,可能无法保证成功。为了解决这些问题,我们提出了一种新的自适应资源索引技术,该技术旨在通过减少每个查询所需的消息和时间来提高搜索的效率和质量。我们的方法由两种互补的技术组成。一种是索引选择技术,它选择性地在每个节点上保留索引,以最小的空间需求增加查询成功的机会。另一种是索引分布技术,它根据搜索性能自动调整索引分布率,以优化搜索性能和开销。我们在各种网络条件下对该技术进行了模拟,结果表明,我们的技术有效地减少了解决查询所需的跳数和消息,并且开销很小。当使用基于泛洪的查询时,即使面对高流失率,它也能将平均跳数减少多达44%,消息减少75%。此外,在有限超时条件下,查询成功率也有所提高,接近100%。
{"title":"Adaptive Resource Indexing Technique for Unstructured Peer-to-Peer Networks","authors":"S. Lerthirunwong, N. Maruyama, S. Matsuoka","doi":"10.1109/CCGRID.2009.41","DOIUrl":"https://doi.org/10.1109/CCGRID.2009.41","url":null,"abstract":"Searching for particular resources in a large-scale decentralized unstructured network can be very difficult since there is no centralized management to provide the specific location of resources. Moreover, the dynamic behavior of networks and the diversity of user behavior cause the search more complex and may not guarantee success. To address the problems, we propose a new adaptive resource indexing technique that aims to increase both efficiency and quality of the search by reducing both messages and time required for each query. Our approach consists of two complementary techniques. One is an index selection technique that selectively keeps the indices at each peer to increase the chance of successful queries with minimum space requirement. Another is an index distribution technique that automatically adjusts index distribution rate based on the search performance to optimize both the search performance and overhead. We simulate the technique in various network conditions and the results show that our technique is effective in decreasing hop counts and messages needed for resolving queries with only small overhead. It decreases the average hop count by up to 44% with 75%-less messages when used with flooding based queries even facing high churn. Furthermore, the query success rate with a limited timeout condition also increases, approaching nearly to 100%.","PeriodicalId":118263,"journal":{"name":"2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124058347","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
AVSS: An Adaptable Virtual Storage System AVSS:一个可适应的虚拟存储系统
Pub Date : 2009-05-18 DOI: 10.1109/CCGRID.2009.42
Jian Ke, Xudong Zhu, Wenwu Na, Lu Xu
This paper presents the design and implementation of the Adaptable Virtual Storage System (AVSS) and introduces the capacity virtualization and performance virtualization for storage systems. AVSS has the following characteristics: 1) adoption of extended YFQ algorithm to control the allocation of bandwidth resources, realization of the performance isolation and guarantees of virtual disks; 2) adoption of hierarchy structure and dynamic mapping mechanism to manage heterogeneous storage resources flexibly and effectively, which lays a foundation for allocating storage resources on demand; 3) application of data-access frequency statistics and dynamic behavior analysis to supervise storage layout reorganization. The experimental results proved the correctness of our design. AVSS can isolate different applications and avoid performance interference. It can adjust storage layout according to the behavior of applications and improve the utilization of storage resources while improving the performance of the storage system.
本文介绍了自适应虚拟存储系统(AVSS)的设计与实现,并介绍了存储系统的容量虚拟化和性能虚拟化。AVSS具有以下特点:1)采用扩展的YFQ算法控制带宽资源的分配,实现虚拟磁盘的性能隔离和保证;2)采用层次结构和动态映射机制,灵活有效地管理异构存储资源,为按需分配存储资源奠定基础;3)应用数据访问频率统计和动态行为分析来监督存储布局重组。实验结果证明了设计的正确性。AVSS可以隔离不同的应用程序,避免性能干扰。它可以根据应用程序的行为调整存储布局,在提高存储系统性能的同时提高存储资源的利用率。
{"title":"AVSS: An Adaptable Virtual Storage System","authors":"Jian Ke, Xudong Zhu, Wenwu Na, Lu Xu","doi":"10.1109/CCGRID.2009.42","DOIUrl":"https://doi.org/10.1109/CCGRID.2009.42","url":null,"abstract":"This paper presents the design and implementation of the Adaptable Virtual Storage System (AVSS) and introduces the capacity virtualization and performance virtualization for storage systems. AVSS has the following characteristics: 1) adoption of extended YFQ algorithm to control the allocation of bandwidth resources, realization of the performance isolation and guarantees of virtual disks; 2) adoption of hierarchy structure and dynamic mapping mechanism to manage heterogeneous storage resources flexibly and effectively, which lays a foundation for allocating storage resources on demand; 3) application of data-access frequency statistics and dynamic behavior analysis to supervise storage layout reorganization. The experimental results proved the correctness of our design. AVSS can isolate different applications and avoid performance interference. It can adjust storage layout according to the behavior of applications and improve the utilization of storage resources while improving the performance of the storage system.","PeriodicalId":118263,"journal":{"name":"2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121473838","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Modeling Job Lifespan Delays in Volunteer Computing Projects 志愿者计算项目中工作寿命延迟的建模
Pub Date : 2009-05-18 DOI: 10.1109/CCGRID.2009.69
Trilce Estrada, M. Taufer, Kevin Reed
Volunteer Computing (VC) projects harness the power of computers owned by volunteers across the Internet to perform hundreds of thousands of independent jobs. In VC projects, the path leading from the generation of jobs to the validation of the job results is characterized by delays hidden in the job lifespan, i.e., distribution delay,in-progress delay, and validation delay. These delays are difficult to estimate because of the dynamic behavior and heterogeneity of VC resources. A wrong estimation of these delays can cause the loss of project throughput and job latency in VC projects. In this paper, we evaluate the accuracy of several probabilistic methods to model the upper time bounds of these delays. We show how our selected models predict up-and-down trends in traces from existing VC projects. The use of our models provides valuable insights on selecting project deadlines and taking scheduling decisions. By accurately predicting job lifespan delays, our models lead to more efficient resource use, higher project throughput, and lower job latency in VC projects.
志愿计算(VC)项目利用互联网上志愿者拥有的计算机的能力来执行数十万个独立的工作。在VC项目中,从作业生成到作业结果验证的过程中,存在隐藏在作业生命周期中的延迟,即分发延迟、进行中延迟和验证延迟。由于VC资源的动态行为和异质性,这些延迟很难估计。对这些延迟的错误估计可能会导致风险投资项目中项目吞吐量和作业延迟的损失。在本文中,我们评估了几种概率方法来模拟这些延迟的上时限的准确性。我们展示了我们选择的模型如何预测来自现有风险投资项目的上升和下降趋势。我们的模型的使用为选择项目截止日期和制定计划决策提供了有价值的见解。通过准确预测作业寿命延迟,我们的模型在VC项目中实现了更有效的资源使用、更高的项目吞吐量和更低的作业延迟。
{"title":"Modeling Job Lifespan Delays in Volunteer Computing Projects","authors":"Trilce Estrada, M. Taufer, Kevin Reed","doi":"10.1109/CCGRID.2009.69","DOIUrl":"https://doi.org/10.1109/CCGRID.2009.69","url":null,"abstract":"Volunteer Computing (VC) projects harness the power of computers owned by volunteers across the Internet to perform hundreds of thousands of independent jobs. In VC projects, the path leading from the generation of jobs to the validation of the job results is characterized by delays hidden in the job lifespan, i.e., distribution delay,in-progress delay, and validation delay. These delays are difficult to estimate because of the dynamic behavior and heterogeneity of VC resources. A wrong estimation of these delays can cause the loss of project throughput and job latency in VC projects. In this paper, we evaluate the accuracy of several probabilistic methods to model the upper time bounds of these delays. We show how our selected models predict up-and-down trends in traces from existing VC projects. The use of our models provides valuable insights on selecting project deadlines and taking scheduling decisions. By accurately predicting job lifespan delays, our models lead to more efficient resource use, higher project throughput, and lower job latency in VC projects.","PeriodicalId":118263,"journal":{"name":"2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122117658","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 36
Multicast Trees for Collaborative Applications 协同应用的组播树
Pub Date : 2009-05-18 DOI: 10.1109/CCGRID.2009.38
K. Rządca, Jackson Tan Teck Yong, Anwitaman Datta
Current implementations of real-time collaborative applications rely on a dedicated infrastructure to carry out all synchronizing and communication functions, and require all end nodes to communicate directly with and through the central server. In this paper, we investigate an architecture, in which the most resource intensive functionality of continuous communication among collaborators to disseminate changes is decentralized, utilizing the end users as relays. We observe that communication characteristics of real-time collaboration makes use of existing multicast mechanisms unsuitable. As collaborative editing sessions are typically long, we are able to gather and then use additional parameters ofnodes (their instabilities and frequency of sending updates) and communication links (latencies and average costs). We identify several criteria to determinethe quality of a multicast tree: cost, latency and instability. We analyze the complexity of these problems and propose algorithms to optimize the communicationtopology. We also consider the multiobjective problem in which we search for a tree that results in a good trade-off between these measures. Validation ofalgorithms on numerous graphs shows that it is important to consider the multiobjective problem, as optimal solutions for one performance measure can be far from optimal values of the others.
实时协作应用程序的当前实现依赖于专用的基础设施来执行所有同步和通信功能,并要求所有终端节点直接与中心服务器通信并通过中心服务器进行通信。在本文中,我们研究了一种架构,在这种架构中,协作者之间持续通信以传播更改的最资源密集型功能是分散的,利用最终用户作为中继。我们观察到实时协作的通信特性使得现有的多播机制不适合使用。由于协作编辑会话通常很长,我们能够收集并使用节点的附加参数(不稳定性和发送更新的频率)和通信链接(延迟和平均成本)。我们确定了几个标准来确定多播树的质量:成本,延迟和不稳定性。我们分析了这些问题的复杂性,并提出了优化通信拓扑的算法。我们还考虑了多目标问题,在这个问题中,我们寻找一棵树,在这些度量之间产生良好的权衡。算法在众多图上的验证表明,考虑多目标问题是很重要的,因为一个性能度量的最优解可能与其他性能度量的最优值相去甚远。
{"title":"Multicast Trees for Collaborative Applications","authors":"K. Rządca, Jackson Tan Teck Yong, Anwitaman Datta","doi":"10.1109/CCGRID.2009.38","DOIUrl":"https://doi.org/10.1109/CCGRID.2009.38","url":null,"abstract":"Current implementations of real-time collaborative applications rely on a dedicated infrastructure to carry out all synchronizing and communication functions, and require all end nodes to communicate directly with and through the central server. In this paper, we investigate an architecture, in which the most resource intensive functionality of continuous communication among collaborators to disseminate changes is decentralized, utilizing the end users as relays. We observe that communication characteristics of real-time collaboration makes use of existing multicast mechanisms unsuitable. As collaborative editing sessions are typically long, we are able to gather and then use additional parameters ofnodes (their instabilities and frequency of sending updates) and communication links (latencies and average costs). We identify several criteria to determinethe quality of a multicast tree: cost, latency and instability. We analyze the complexity of these problems and propose algorithms to optimize the communicationtopology. We also consider the multiobjective problem in which we search for a tree that results in a good trade-off between these measures. Validation ofalgorithms on numerous graphs shows that it is important to consider the multiobjective problem, as optimal solutions for one performance measure can be far from optimal values of the others.","PeriodicalId":118263,"journal":{"name":"2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid","volume":"185 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2009-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124701283","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
期刊
2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1