I. Terekhov, R. Pordes, V. White, L. Lueking, L. Loebel-Carpenter, J. Trumbo, S. Veseli, M. Vranicar, S. White, H. Schellman
{"title":"Distributed data access and resource management in the D0 SAM system","authors":"I. Terekhov, R. Pordes, V. White, L. Lueking, L. Loebel-Carpenter, J. Trumbo, S. Veseli, M. Vranicar, S. White, H. Schellman","doi":"10.1109/HPDC.2001.945179","DOIUrl":null,"url":null,"abstract":"SAM (Sequential Access through Meta-data) is the data access and job management system for the D0 high energy physics experiment at Fermilab. The SAM system is being developed and used to handle the Petabyte-scale experiment data, accessed by hundreds of D0 collaborators scattered around the world. In this paper, we present solutions to some of the distributed data processing problems from the perspective of real experience dealing with mission-critical data. We concentrate on the distributed disk caching, resource management and job control. The system has elements of Grid computing and has features applicable to data-intensive computing in general.","PeriodicalId":304683,"journal":{"name":"Proceedings 10th IEEE International Symposium on High Performance Distributed Computing","volume":"46 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2001-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"32","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings 10th IEEE International Symposium on High Performance Distributed Computing","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/HPDC.2001.945179","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 32
Abstract
SAM (Sequential Access through Meta-data) is the data access and job management system for the D0 high energy physics experiment at Fermilab. The SAM system is being developed and used to handle the Petabyte-scale experiment data, accessed by hundreds of D0 collaborators scattered around the world. In this paper, we present solutions to some of the distributed data processing problems from the perspective of real experience dealing with mission-critical data. We concentrate on the distributed disk caching, resource management and job control. The system has elements of Grid computing and has features applicable to data-intensive computing in general.
SAM (Sequential Access through Meta-data)是Fermilab D0高能物理实验的数据访问和作业管理系统。SAM系统正在开发中，并用于处理分布在世界各地的数百名D0合作者访问的pb级实验数据。在本文中，我们从处理关键任务数据的实际经验的角度提出了一些分布式数据处理问题的解决方案。我们专注于分布式磁盘缓存、资源管理和作业控制。该系统具有网格计算的元素，具有适用于一般数据密集型计算的特性。
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
