首页 > 最新文献

Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.最新文献

英文 中文
Using passive traces of application traffic in a network monitoring system 在网络监控系统中使用被动跟踪应用程序流量
M. Zangrilli, B. Lowekamp
Adaptive grid applications require up-to-date network resource measurements and predictions to help steer their adaptation to meet performance goals. To this end, we are interested in monitoring the available bandwidth of the underlying networks in the most accurate and least obtrusive way. Bandwidth is either measured by actively injecting data probes into the network or by passively monitoring existing traffic, but there is a definite trade-off between the active approach, which is invasive, and the passive approach, which is rendered ineffective during periods of network idleness. We are developing the Wren bandwidth monitoring tool, which uses packet traces of existing application traffic to measure available bandwidth. We demonstrate that the principles supporting active bandwidth tools can be applied to passive traces of the LAN and WAN traffic generated by high-performance grid applications. We use our results to form a preliminary characterization of the application traffic required by available bandwidth techniques to produce effective measurements. Our results indicate that a low overhead, passive monitoring system supplemented with active measurements can be built to obtain a complete picture of the network's performance.
自适应网格应用程序需要最新的网络资源测量和预测,以帮助引导其适应以满足性能目标。为此,我们感兴趣的是以最准确和最不突兀的方式监控底层网络的可用带宽。带宽要么通过主动地向网络注入数据探测来测量,要么通过被动地监视现有流量来测量,但是在主动方法(具有侵入性)和被动方法(在网络空闲期间无效)之间存在明确的权衡。我们正在开发Wren带宽监控工具,它使用现有应用程序流量的数据包跟踪来测量可用带宽。我们证明了支持主动带宽工具的原理可以应用于高性能网格应用程序生成的LAN和WAN流量的被动跟踪。我们使用我们的结果来形成可用带宽技术所需的应用程序流量的初步特征,以产生有效的测量。我们的研究结果表明,可以建立一个低开销的被动监测系统,辅以主动测量,以获得网络性能的完整图像。
{"title":"Using passive traces of application traffic in a network monitoring system","authors":"M. Zangrilli, B. Lowekamp","doi":"10.1109/HPDC.2004.38","DOIUrl":"https://doi.org/10.1109/HPDC.2004.38","url":null,"abstract":"Adaptive grid applications require up-to-date network resource measurements and predictions to help steer their adaptation to meet performance goals. To this end, we are interested in monitoring the available bandwidth of the underlying networks in the most accurate and least obtrusive way. Bandwidth is either measured by actively injecting data probes into the network or by passively monitoring existing traffic, but there is a definite trade-off between the active approach, which is invasive, and the passive approach, which is rendered ineffective during periods of network idleness. We are developing the Wren bandwidth monitoring tool, which uses packet traces of existing application traffic to measure available bandwidth. We demonstrate that the principles supporting active bandwidth tools can be applied to passive traces of the LAN and WAN traffic generated by high-performance grid applications. We use our results to form a preliminary characterization of the application traffic required by available bandwidth techniques to produce effective measurements. Our results indicate that a low overhead, passive monitoring system supplemented with active measurements can be built to obtain a complete picture of the network's performance.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127653265","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 23
Evaluation of distributed recovery in large-scale storage systems 大规模存储系统分布式恢复评估
Qin Xin, E. L. Miller, T. Schwarz
Storage clusters consisting of thousands of disk drives are now being used both for their large capacity and high throughput. However, their reliability is far worse than that of smaller storage systems due to the increased number of storage nodes. RAID technology is no longer sufficient to guarantee the necessary high data reliability for such systems, because disk rebuild time lengthens as disk capacity grows. We present fast recovery mechanism (FARM), a distributed recovery approach that exploits excess disk capacity and reduces data recovery time. FARM works in concert with replication and erasure-coding redundancy schemes to dramatically lower the probability of data loss in large-scale storage systems. We have examined essential factors that influence system reliability, performance, and costs, such as failure detections, disk bandwidth usage for recovery, disk space utilization, disk drive replacement, and system scales, by simulating system behavior under disk failures. Our results show the reliability improvement from FARM and demonstrate the impacts of various factors on system reliability. Using our techniques, system designers will be better able to build multipetabyte storage systems with much higher reliability at lower cost than previously possible.
由数千个磁盘驱动器组成的存储集群现在被用于大容量和高吞吐量。但由于存储节点数量的增加,其可靠性远不如小型存储系统。RAID技术不再足以保证此类系统所需的高数据可靠性,因为随着磁盘容量的增加,磁盘重建时间也会延长。我们提出了快速恢复机制(FARM),这是一种利用多余磁盘容量并减少数据恢复时间的分布式恢复方法。FARM与复制和擦除编码冗余方案协同工作,可以显著降低大型存储系统中数据丢失的概率。通过模拟磁盘故障下的系统行为,我们研究了影响系统可靠性、性能和成本的基本因素,例如故障检测、用于恢复的磁盘带宽使用、磁盘空间利用、磁盘驱动器替换和系统规模。我们的结果显示了FARM对系统可靠性的改善,并展示了各种因素对系统可靠性的影响。使用我们的技术,系统设计人员将能够以比以前更低的成本更好地构建可靠性更高的多拍字节存储系统。
{"title":"Evaluation of distributed recovery in large-scale storage systems","authors":"Qin Xin, E. L. Miller, T. Schwarz","doi":"10.1109/HPDC.2004.12","DOIUrl":"https://doi.org/10.1109/HPDC.2004.12","url":null,"abstract":"Storage clusters consisting of thousands of disk drives are now being used both for their large capacity and high throughput. However, their reliability is far worse than that of smaller storage systems due to the increased number of storage nodes. RAID technology is no longer sufficient to guarantee the necessary high data reliability for such systems, because disk rebuild time lengthens as disk capacity grows. We present fast recovery mechanism (FARM), a distributed recovery approach that exploits excess disk capacity and reduces data recovery time. FARM works in concert with replication and erasure-coding redundancy schemes to dramatically lower the probability of data loss in large-scale storage systems. We have examined essential factors that influence system reliability, performance, and costs, such as failure detections, disk bandwidth usage for recovery, disk space utilization, disk drive replacement, and system scales, by simulating system behavior under disk failures. Our results show the reliability improvement from FARM and demonstrate the impacts of various factors on system reliability. Using our techniques, system designers will be better able to build multipetabyte storage systems with much higher reliability at lower cost than previously possible.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124460231","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 93
An empirical study of the multiscale predictability of network traffic 网络流量多尺度可预测性的实证研究
Y. Qiao, J. Skicewicz, P. Dinda
Distributed applications use predictions of network traffic to sustain their performance by adapting their behavior The timescale of interest is application-dependent and thus it is natural to ask how predictability depends on the resolution, or degree of smoothing, of the network traffic signal. To help answer this question we empirically study the one-step-ahead predictability, measured by the ratio of mean squared error to signal variance, of network traffic at different resolutions. A one-step-ahead prediction at a coarse resolution is a prediction of the average behavior over a long interval We apply a wide range of linear and nonlinear time series models to a large number of packet traces, generating different resolution views of the traces through two methods: the simple binning approach used by several extant network measurement tools, and by wavelet-based approximations. The wavelet-based approach is a natural way to provide multiscale prediction to applications. We find that predictability seems to be highly situational in practice - it varies widely from trace to trace. Unexpectedly, predictability does not always increase as the signal is smoothed. Half of the time there is a sweet spot at which the ratio is minimized and predictability is clearly the best. Also surprisingly, predictors that can capture non-stationarity and nonlinearity provide benefits only at very coarse resolutions.
分布式应用程序使用网络流量的预测来通过调整其行为来维持其性能。感兴趣的时间尺度依赖于应用程序,因此很自然地要问可预测性如何依赖于网络流量信号的分辨率或平滑程度。为了帮助回答这个问题,我们实证研究了一步前的可预测性,用均方误差与信号方差的比值来衡量,在不同分辨率下的网络流量。我们将广泛的线性和非线性时间序列模型应用于大量的数据包路径,通过两种方法生成不同的路径分辨率视图:几种现有网络测量工具使用的简单分箱方法和基于小波的近似。基于小波的方法是为应用提供多尺度预测的自然方法。我们发现,在实践中,可预测性似乎是高度情境化的——它在每个线索之间都有很大的差异。出乎意料的是,随着信号的平滑,可预测性并不总是增加。有一半的时间存在一个最佳点,在这个点上,比例最小,可预测性显然是最好的。同样令人惊讶的是,能够捕捉非平稳性和非线性的预测器只能在非常粗糙的分辨率下提供好处。
{"title":"An empirical study of the multiscale predictability of network traffic","authors":"Y. Qiao, J. Skicewicz, P. Dinda","doi":"10.1109/HPDC.2004.3","DOIUrl":"https://doi.org/10.1109/HPDC.2004.3","url":null,"abstract":"Distributed applications use predictions of network traffic to sustain their performance by adapting their behavior The timescale of interest is application-dependent and thus it is natural to ask how predictability depends on the resolution, or degree of smoothing, of the network traffic signal. To help answer this question we empirically study the one-step-ahead predictability, measured by the ratio of mean squared error to signal variance, of network traffic at different resolutions. A one-step-ahead prediction at a coarse resolution is a prediction of the average behavior over a long interval We apply a wide range of linear and nonlinear time series models to a large number of packet traces, generating different resolution views of the traces through two methods: the simple binning approach used by several extant network measurement tools, and by wavelet-based approximations. The wavelet-based approach is a natural way to provide multiscale prediction to applications. We find that predictability seems to be highly situational in practice - it varies widely from trace to trace. Unexpectedly, predictability does not always increase as the signal is smoothed. Half of the time there is a sweet spot at which the ratio is minimized and predictability is clearly the best. Also surprisingly, predictors that can capture non-stationarity and nonlinearity provide benefits only at very coarse resolutions.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121013335","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 77
Distributed file system support for virtual machines in grid computing 网格计算中虚拟机的分布式文件系统支持
Ming Zhao, Jian Zhang, Renato J. O. Figueiredo
This paper presents a data management solution which allows fast virtual machine (VM) instantiation and efficient run-time execution to support VMs as execution environments in grid computing. It is based on novel distributed file system virtualization techniques and is unique in that: 1) it provides on-demand access to VM state for unmodified VM monitors; 2) it supports user-level and write-back disk caches, per-application caching policies and middleware-driven consistency models; and 3) it supports the use of meta-data associated with files to expedite data transfers. The paper reports on its performance in a WAN setup using VMware-based VMs. Results show that the solution delivers performance over 30% better than native NFS and can bring application-perceived overheads below 10% relatively to a local disk setup. The solution also allows a VM with 1.6GB virtual disk and 320MB virtual memory to be cloned within 160 seconds when it is first instantiated (and within 25 seconds for subsequent clones).
本文提出了一种数据管理解决方案,该方案允许快速的虚拟机实例化和高效的运行时执行,以支持虚拟机作为网格计算中的执行环境。它基于新颖的分布式文件系统虚拟化技术,其独特之处在于:1)它为未修改的VM监视器提供对VM状态的按需访问;2)支持用户级和回写磁盘缓存、应用级缓存策略和中间件驱动的一致性模型;3)它支持使用与文件关联的元数据来加快数据传输。本文报告了它在使用基于vmware的虚拟机的广域网设置中的性能。结果表明,该解决方案提供的性能比本地NFS好30%以上,并且相对于本地磁盘设置,可以将应用程序感知到的开销降低到10%以下。该解决方案还允许具有1.6GB虚拟磁盘和320MB虚拟内存的虚拟机在首次实例化时在160秒内被克隆(后续克隆在25秒内)。
{"title":"Distributed file system support for virtual machines in grid computing","authors":"Ming Zhao, Jian Zhang, Renato J. O. Figueiredo","doi":"10.1109/HPDC.2004.10","DOIUrl":"https://doi.org/10.1109/HPDC.2004.10","url":null,"abstract":"This paper presents a data management solution which allows fast virtual machine (VM) instantiation and efficient run-time execution to support VMs as execution environments in grid computing. It is based on novel distributed file system virtualization techniques and is unique in that: 1) it provides on-demand access to VM state for unmodified VM monitors; 2) it supports user-level and write-back disk caches, per-application caching policies and middleware-driven consistency models; and 3) it supports the use of meta-data associated with files to expedite data transfers. The paper reports on its performance in a WAN setup using VMware-based VMs. Results show that the solution delivers performance over 30% better than native NFS and can bring application-perceived overheads below 10% relatively to a local disk setup. The solution also allows a VM with 1.6GB virtual disk and 320MB virtual memory to be cloned within 160 seconds when it is first instantiated (and within 25 seconds for subsequent clones).","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125697482","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 42
SpiderNet: an integrated peer-to-peer service composition framework SpiderNet:集成的点对点服务组合框架
Xiaohui Gu, K. Nahrstedt, Bin Yu
Service composition is highly desirable in peer-to-peer (P2P) systems where application services are naturally dispersed on distributed peers. However, it is challenging to provide high quality and failure resilient service composition in P2P systems due to the decentralization requirement and dynamic peer arrivals/departures. We present an integrated P2P service composition framework called SpiderNet to address the challenges. At service setup phase, SpiderNet performs a novel bounded composition probing protocol to provide scalable quality-aware and resource-efficient sendee composition in a fully distributed fashion. Moreover, SpiderNet supports directed acyclic graph composition topologies and explores exchangeable composition orders for enhanced service quality. During service runtime, SpiderNet provides proactive failure recovery to overcome dynamic changes (e.g., peer departures) in P2P systems. The proactive failure recovery scheme maintains a small number of dynamically selected backup compositions to achieve quick failure recovery for soft realtime streaming applications. We have implemented a prototype of SpiderNet and conducted extensive experiments using both large-scale simulations and wide-area network testbed. Experimental results show the feasibility and efficiency of the SpiderNet service composition solution for P2P systems.
在点对点(P2P)系统中,应用程序服务自然地分散在分布式的对等点上,服务组合是非常理想的。然而,由于去中心化需求和动态的对等点到达/离开,在P2P系统中提供高质量和故障弹性的服务组合是具有挑战性的。我们提出了一个名为SpiderNet的集成P2P服务组合框架来解决这些挑战。在服务设置阶段,SpiderNet执行一种新颖的有界组合探测协议,以完全分布式的方式提供可扩展的质量感知和资源高效的发送者组合。此外,SpiderNet支持有向无循环图组合拓扑,并探索可交换的组合顺序,以提高服务质量。在服务运行期间,SpiderNet提供主动故障恢复,以克服P2P系统中的动态变化(例如,对等点偏离)。主动故障恢复方案维护少量动态选择的备份组合,以实现软实时流应用的快速故障恢复。我们已经实现了SpiderNet的原型,并使用大规模模拟和广域网测试平台进行了广泛的实验。实验结果表明了SpiderNet服务组合方案在P2P系统中的可行性和有效性。
{"title":"SpiderNet: an integrated peer-to-peer service composition framework","authors":"Xiaohui Gu, K. Nahrstedt, Bin Yu","doi":"10.1109/HPDC.2004.32","DOIUrl":"https://doi.org/10.1109/HPDC.2004.32","url":null,"abstract":"Service composition is highly desirable in peer-to-peer (P2P) systems where application services are naturally dispersed on distributed peers. However, it is challenging to provide high quality and failure resilient service composition in P2P systems due to the decentralization requirement and dynamic peer arrivals/departures. We present an integrated P2P service composition framework called SpiderNet to address the challenges. At service setup phase, SpiderNet performs a novel bounded composition probing protocol to provide scalable quality-aware and resource-efficient sendee composition in a fully distributed fashion. Moreover, SpiderNet supports directed acyclic graph composition topologies and explores exchangeable composition orders for enhanced service quality. During service runtime, SpiderNet provides proactive failure recovery to overcome dynamic changes (e.g., peer departures) in P2P systems. The proactive failure recovery scheme maintains a small number of dynamically selected backup compositions to achieve quick failure recovery for soft realtime streaming applications. We have implemented a prototype of SpiderNet and conducted extensive experiments using both large-scale simulations and wide-area network testbed. Experimental results show the feasibility and efficiency of the SpiderNet service composition solution for P2P systems.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125955917","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 170
Showcasing the features and capabilities of NEESgrid: a grid based system for the earthquake engineering domain 展示了NEESgrid的特性和功能:一个基于网格的地震工程领域系统
S. Gullapalli, S. Dyke, P. Hubbard, Doru Marcusiu, L. Pearlman, C. Severance
The NEESgrid project is revolutionizing the way earthquake engineering (EE) researchers collaborate to plan, perform, share and publish research even while being geographically separated. The proposed demonstration is in complement to the NEESgrid paper titled "Distributed Hybrid Earthquake Engineering Experiments: Experiences with a Ground-Shaking Grid Application" also presented at this conference. It showcases some of NEESgrid"s features and capabilities in the area of distributed hybrid experiments, as well as new features developed since the distributed hybrid experiment was conducted. These new features and capabilities are in the areas of simulation portal, electronic notebook, data modeling, and improved live streaming of near real time sensor and video data.
NEESgrid项目正在彻底改变地震工程(EE)研究人员协作规划、执行、共享和发表研究成果的方式,即使他们在地理上是分开的。该提议的演示是对NEESgrid题为“分布式混合地震工程实验:震动网格应用的经验”的论文的补充,该论文也在本次会议上发表。它展示了NEESgrid在分布式混合实验领域的一些特性和功能,以及自进行分布式混合实验以来开发的新特性。这些新特性和功能涉及模拟门户、电子笔记本、数据建模以及改进的近实时传感器和视频数据的实时流。
{"title":"Showcasing the features and capabilities of NEESgrid: a grid based system for the earthquake engineering domain","authors":"S. Gullapalli, S. Dyke, P. Hubbard, Doru Marcusiu, L. Pearlman, C. Severance","doi":"10.1109/HPDC.2004.31","DOIUrl":"https://doi.org/10.1109/HPDC.2004.31","url":null,"abstract":"The NEESgrid project is revolutionizing the way earthquake engineering (EE) researchers collaborate to plan, perform, share and publish research even while being geographically separated. The proposed demonstration is in complement to the NEESgrid paper titled \"Distributed Hybrid Earthquake Engineering Experiments: Experiences with a Ground-Shaking Grid Application\" also presented at this conference. It showcases some of NEESgrid\"s features and capabilities in the area of distributed hybrid experiments, as well as new features developed since the distributed hybrid experiment was conducted. These new features and capabilities are in the areas of simulation portal, electronic notebook, data modeling, and improved live streaming of near real time sensor and video data.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"98 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127330579","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
Globus and PlanetLab resource management solutions compared Globus和PlanetLab资源管理解决方案的比较
M. Ripeanu, M. Bowman, J. Chase, Ian T Foster, Milan Milenkovic
PlanetLab and Globus Toolkit are gaining widespread adoption in their respective communities. Although designed to solve different problems - PlanetLab is deploying a worldwide infrastructure testbed for experimenting with network services, while Globus is offering general, standards-based, software for running distributed applications over aggregated, shared resources - both build infrastructures that enable federated, extensible, and secure resource sharing across trust domains. Thus, it is instructive to compare their resource management solutions. To this end, we review the approaches taken in the two systems, attempt to trace back to starting assumptions the differences in these approaches, and explore scenarios where the two platforms can cooperate to the benefit of both user communities. We believe that this is a key first step to identifying pieces that could be shared by the two communities, pieces that are complementary, and how Globus and PlanetLab might ultimately evolve together.
PlanetLab和Globus Toolkit正在各自的社区中获得广泛采用。PlanetLab是为了解决不同的问题而设计的——PlanetLab正在部署一个全球性的基础设施测试平台,用于试验网络服务,而Globus则提供通用的、基于标准的软件,用于在聚合的、共享的资源上运行分布式应用程序——两者都构建了能够跨信任域实现联合的、可扩展的和安全的资源共享的基础设施。因此,比较它们的资源管理解决方案是有指导意义的。为此,我们回顾了两个系统中采用的方法,试图追溯到这些方法的初始假设差异,并探索两个平台可以合作以造福两个用户社区的场景。我们相信,这是关键的第一步,可以确定两个社区可以共享的部分,互补的部分,以及Globus和PlanetLab最终如何共同发展。
{"title":"Globus and PlanetLab resource management solutions compared","authors":"M. Ripeanu, M. Bowman, J. Chase, Ian T Foster, Milan Milenkovic","doi":"10.1109/HPDC.2004.17","DOIUrl":"https://doi.org/10.1109/HPDC.2004.17","url":null,"abstract":"PlanetLab and Globus Toolkit are gaining widespread adoption in their respective communities. Although designed to solve different problems - PlanetLab is deploying a worldwide infrastructure testbed for experimenting with network services, while Globus is offering general, standards-based, software for running distributed applications over aggregated, shared resources - both build infrastructures that enable federated, extensible, and secure resource sharing across trust domains. Thus, it is instructive to compare their resource management solutions. To this end, we review the approaches taken in the two systems, attempt to trace back to starting assumptions the differences in these approaches, and explore scenarios where the two platforms can cooperate to the benefit of both user communities. We believe that this is a key first step to identifying pieces that could be shared by the two communities, pieces that are complementary, and how Globus and PlanetLab might ultimately evolve together.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"75 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115035657","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 25
Performance and scalability of a replica location service 副本位置服务的性能和可伸缩性
A. Chervenak, Naveen Palavalli, S. Bharathi, C. Kesselman, Robert Schwartzkopf
We describe the implementation and evaluate the performance of a replica location service that is part of the Globus Toolkit Version 3.0. A replica location service (RLS) provides a mechanism for registering the existence of replicas and discovering them. Features of our implementation include the use of soft state update protocols to populate a distributed index and optional Bloom filter compression to reduce the size of these updates. Our results demonstrate that RLS performance scales well for individual servers with millions of entries and up to 100 requesting threads. We also show that the distributed RLS index scales well when using Bloom filter compression for wide area updates.
我们将描述作为Globus Toolkit Version 3.0一部分的副本位置服务的实现并评估其性能。副本位置服务(RLS)提供了一种机制,用于注册副本的存在并发现它们。我们实现的功能包括使用软状态更新协议来填充分布式索引和可选的Bloom过滤器压缩来减少这些更新的大小。我们的结果表明,对于具有数百万条目和多达100个请求线程的单个服务器,RLS性能可以很好地扩展。我们还表明,当使用布隆过滤器压缩进行广域更新时,分布式RLS索引可以很好地扩展。
{"title":"Performance and scalability of a replica location service","authors":"A. Chervenak, Naveen Palavalli, S. Bharathi, C. Kesselman, Robert Schwartzkopf","doi":"10.1109/HPDC.2004.27","DOIUrl":"https://doi.org/10.1109/HPDC.2004.27","url":null,"abstract":"We describe the implementation and evaluate the performance of a replica location service that is part of the Globus Toolkit Version 3.0. A replica location service (RLS) provides a mechanism for registering the existence of replicas and discovering them. Features of our implementation include the use of soft state update protocols to populate a distributed index and optional Bloom filter compression to reduce the size of these updates. Our results demonstrate that RLS performance scales well for individual servers with millions of entries and up to 100 requesting threads. We also show that the distributed RLS index scales well when using Bloom filter compression for wide area updates.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122504627","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 185
GATES: a grid-based middleware for processing distributed data streams GATES:用于处理分布式数据流的基于网格的中间件
Liang Chen, Kolagatla Reddy, G. Agrawal
Increasingly, a number of applications rely on, or can potentially benefit from, analysis and monitoring of data streams. Moreover, many of these applications involve high volume data streams and require distributed processing of data arising from a distributed set of sources. Thus, we believe that a grid environment is well suited for flexible and adaptive analysis of these streams. This paper reports the design and initial evaluation of a middleware for processing distributed data streams. Our system is referred to as GATES (grid-based adaptive execution on streams). This system is designed to use the existing grid standards and tools to the extent possible. It flexibly achieves the best accuracy that is possible while maintaining the real-time constraint on the analysis. We have developed a self-adaptation algorithm for this purpose. Results from a detailed evaluation of this system demonstrate the benefits of distributed processing, and the effectiveness of our self-adaptation algorithm.
越来越多的应用程序依赖于或可能受益于数据流的分析和监控。此外,许多此类应用程序涉及大容量数据流,需要对来自分布式源集的数据进行分布式处理。因此,我们认为网格环境非常适合对这些流进行灵活和自适应的分析。本文报道了一个用于处理分布式数据流的中间件的设计和初步评估。我们的系统被称为GATES(基于网格的自适应流执行)。本系统旨在尽可能使用现有的网格标准和工具。它灵活地在保持分析的实时约束的同时实现可能的最佳精度。我们为此开发了一种自适应算法。对该系统的详细评估结果表明了分布式处理的优点,以及我们的自适应算法的有效性。
{"title":"GATES: a grid-based middleware for processing distributed data streams","authors":"Liang Chen, Kolagatla Reddy, G. Agrawal","doi":"10.1109/HPDC.2004.16","DOIUrl":"https://doi.org/10.1109/HPDC.2004.16","url":null,"abstract":"Increasingly, a number of applications rely on, or can potentially benefit from, analysis and monitoring of data streams. Moreover, many of these applications involve high volume data streams and require distributed processing of data arising from a distributed set of sources. Thus, we believe that a grid environment is well suited for flexible and adaptive analysis of these streams. This paper reports the design and initial evaluation of a middleware for processing distributed data streams. Our system is referred to as GATES (grid-based adaptive execution on streams). This system is designed to use the existing grid standards and tools to the extent possible. It flexibly achieves the best accuracy that is possible while maintaining the real-time constraint on the analysis. We have developed a self-adaptation algorithm for this purpose. Results from a detailed evaluation of this system demonstrate the benefits of distributed processing, and the effectiveness of our self-adaptation algorithm.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122339326","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 70
UT Grid: a comprehensive campus cyberinfrastructure UT网格:一个全面的校园网络基础设施
J. Boisseau
UT Grid is a comprehensive campus cyberinfrastructure project to integrate the numerous and diverse computational, visualization, storage, data and information, and instrument/device resources of The University of Texas at Austin (UT). This joint project between UT Austin and IBM has a focus and approach with important fundamental differences from multiinstitution grids and discipline-specific grids. These distinctions, coupled with new locally-developed software for providing both portal and shell-based user interfaces to numerous grid software technologies, facilitate rapid deployment, adoption, and evolution of UT Grid, while enabling it to serve as a platform for both production computing (for research and education) and grid computing research. The first stages of UT grid are well under way after only two months: the construction of grid user portals and grid user nodes as interfaces, and the integration of serial and parallel computing resources for high-throughput computing.
德克萨斯大学奥斯汀分校网格是一个综合性的校园网络基础设施项目,旨在整合德克萨斯大学奥斯汀分校(UT)众多不同的计算、可视化、存储、数据和信息以及仪器/设备资源。UT Austin和IBM之间的这个联合项目的重点和方法与多机构网格和特定学科网格有重要的根本区别。这些区别,加上新的本地开发的软件,为众多网格软件技术提供门户和基于shell的用户界面,促进了UT网格的快速部署、采用和发展,同时使其能够作为生产计算(用于研究和教育)和网格计算研究的平台。UT网格的第一阶段仅用了两个月的时间就顺利展开:构建网格用户门户和网格用户节点作为接口,整合串行和并行计算资源,实现高吞吐量计算。
{"title":"UT Grid: a comprehensive campus cyberinfrastructure","authors":"J. Boisseau","doi":"10.1109/HPDC.2004.39","DOIUrl":"https://doi.org/10.1109/HPDC.2004.39","url":null,"abstract":"UT Grid is a comprehensive campus cyberinfrastructure project to integrate the numerous and diverse computational, visualization, storage, data and information, and instrument/device resources of The University of Texas at Austin (UT). This joint project between UT Austin and IBM has a focus and approach with important fundamental differences from multiinstitution grids and discipline-specific grids. These distinctions, coupled with new locally-developed software for providing both portal and shell-based user interfaces to numerous grid software technologies, facilitate rapid deployment, adoption, and evolution of UT Grid, while enabling it to serve as a platform for both production computing (for research and education) and grid computing research. The first stages of UT grid are well under way after only two months: the construction of grid user portals and grid user nodes as interfaces, and the integration of serial and parallel computing resources for high-throughput computing.","PeriodicalId":446429,"journal":{"name":"Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.","volume":"58 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117250418","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
期刊
Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004.
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1