Launcher: A Shell-based Framework for Rapid Development of Parallel Parametric Studies

Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.) Pub Date : 2014-07-13 DOI:10.1145/2616498.2616534

Lucas A. Wilson, John M. Fonner

{"title":"Launcher: A Shell-based Framework for Rapid Development of Parallel Parametric Studies","authors":"Lucas A. Wilson, John M. Fonner","doi":"10.1145/2616498.2616534","DOIUrl":null,"url":null,"abstract":"Petascale computing systems have enabled tremendous advances for traditional simulation and modeling algorithms that are built around parallel execution. Unfortunately, scientific domains using data-oriented or high-throughput paradigms have difficulty taking full advantage of these resources without custom software development. This paper describes our solution for rapidly developing parallel parametric studies using sequential or threaded tasks: The launcher. We detail how to get ensembles executing quickly through common job schedulers SGE and SLURM, and the various user-customizable options that the launcher provides. We illustrate the efficiency of or tool by presenting execution results at large scale (over 65,000 cores) for varying workloads, including a virtual screening workload with indeterminate runtimes using the drug docking software Autodock Vina.","PeriodicalId":93364,"journal":{"name":"Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)","volume":"146 1","pages":"40:1-40:8"},"PeriodicalIF":0.0000,"publicationDate":"2014-07-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"16","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/2616498.2616534","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 16

Abstract

Petascale computing systems have enabled tremendous advances for traditional simulation and modeling algorithms that are built around parallel execution. Unfortunately, scientific domains using data-oriented or high-throughput paradigms have difficulty taking full advantage of these resources without custom software development. This paper describes our solution for rapidly developing parallel parametric studies using sequential or threaded tasks: The launcher. We detail how to get ensembles executing quickly through common job schedulers SGE and SLURM, and the various user-customizable options that the launcher provides. We illustrate the efficiency of or tool by presenting execution results at large scale (over 65,000 cores) for varying workloads, including a virtual screening workload with indeterminate runtimes using the drug docking software Autodock Vina.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

启动器:一个基于shell的框架，用于并行参数化研究的快速发展

千兆级计算系统使围绕并行执行构建的传统仿真和建模算法取得了巨大进步。不幸的是，如果没有定制软件开发，使用面向数据或高吞吐量范例的科学领域很难充分利用这些资源。本文描述了我们使用顺序或线程任务快速开发并行参数研究的解决方案:启动器。我们详细介绍了如何通过通用作业调度程序SGE和SLURM快速执行集成，以及启动程序提供的各种用户可自定义选项。我们通过展示针对不同工作负载(包括使用药物对接软件Autodock Vina具有不确定运行时间的虚拟筛选工作负载)的大规模(超过65,000个核)执行结果来说明该工具的效率。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Proceedings of XSEDE16 : Diversity, Big Data, and Science at Scale : July 17-21, 2016, Intercontinental Miami Hotel, Miami, Florida, USA. Conference on Extreme Science and Engineering Discovery Environment (5th : 2016 : Miami, Fla.)

自引率

0.00%

发文量

期刊最新文献

CloudBridge: a Simple Cross-Cloud Python Library. pbsacct: A Workload Analysis System for PBS-Based HPC Systems ECSS Experience: Particle Tracing Reinvented Fast, Low-Memory Algorithm for Construction of Nanosecond Level Snapshots of Financial Markets Benchmarking SSD-Based Lustre File System Configurations