Fast Fourier Transform (FFT) is one of the most important numerical algorithms widely used in numerous scientific and engineering computations. With the emergence of big data problems, however, it is challenging to acquire, process and store a sufficient amount of data to compute the FFT in the first place. Recently developed sparse FFT (sFFT) algorithm provides a solution to this problem. sFFT computes a compressed Fourier transform by using only a small subset of the input data, thus achieving significant performance improvement. While the increase in the number of cores and memory bandwidth on modern architectures provide an opportunity to improve the performance through sophisticated parallel algorithm design, sFFT is inherently complex, and numerous challenges need to be addressed. Among all the challenges, sFFT falls into the category of irregular applications in which memory access patterns are indirect and irregular that exhibit poor data locality. In this paper, we explore data layout transformation algorithms to tackle the challenge. Our approach shows that an optimized and locality-aware parallel sFFT can perform 7x faster than the original sequential sFFT library on a multicore platform. This optimized locality-aware parallel sFFT is also approximately 10x faster than the parallel FFTW.
{"title":"An Efficient Data Layout Transformation Algorithm for Locality-Aware Parallel Sparse FFT","authors":"Cheng Wang, S. Chandrasekaran, B. Chapman","doi":"10.1145/3149704.3149769","DOIUrl":"https://doi.org/10.1145/3149704.3149769","url":null,"abstract":"Fast Fourier Transform (FFT) is one of the most important numerical algorithms widely used in numerous scientific and engineering computations. With the emergence of big data problems, however, it is challenging to acquire, process and store a sufficient amount of data to compute the FFT in the first place. Recently developed sparse FFT (sFFT) algorithm provides a solution to this problem. sFFT computes a compressed Fourier transform by using only a small subset of the input data, thus achieving significant performance improvement. While the increase in the number of cores and memory bandwidth on modern architectures provide an opportunity to improve the performance through sophisticated parallel algorithm design, sFFT is inherently complex, and numerous challenges need to be addressed. Among all the challenges, sFFT falls into the category of irregular applications in which memory access patterns are indirect and irregular that exhibit poor data locality. In this paper, we explore data layout transformation algorithms to tackle the challenge. Our approach shows that an optimized and locality-aware parallel sFFT can perform 7x faster than the original sequential sFFT library on a multicore platform. This optimized locality-aware parallel sFFT is also approximately 10x faster than the parallel FFTW.","PeriodicalId":292798,"journal":{"name":"Proceedings of the Seventh Workshop on Irregular Applications: Architectures and Algorithms","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128079161","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The Intel Knight Landing (KNL) manycore chip includes 3D-stacked memory named MCDRAM, also known as High Bandwidth Memory (HBM) for parallel applications that needs to scale to high thread count. In this paper, we provide a quantitative study of the KNL for HPC proxy applications including Lulesh, HPCG, AMG, and Hotspot when using DDR4 and MCDRAM. The results indicate that HBM significantly improves the performance of memory intensive applications for as many as three times better than DDR4 in HPCG, and Lulesh and HPCG for as many as 40% and 200%. For the selected compute intensive applications, the performance advantage of MCDRAM over DDR4 varies from 2% to 28%. We also observed that the cross-points, where MCDRAM starts outperforming DDR4, are around 8 to 16 threads.
{"title":"Evaluation of Knight Landing High Bandwidth Memory for HPC Workloads","authors":"S. Salehian, Yonghong Yan","doi":"10.1145/3149704.3149766","DOIUrl":"https://doi.org/10.1145/3149704.3149766","url":null,"abstract":"The Intel Knight Landing (KNL) manycore chip includes 3D-stacked memory named MCDRAM, also known as High Bandwidth Memory (HBM) for parallel applications that needs to scale to high thread count. In this paper, we provide a quantitative study of the KNL for HPC proxy applications including Lulesh, HPCG, AMG, and Hotspot when using DDR4 and MCDRAM. The results indicate that HBM significantly improves the performance of memory intensive applications for as many as three times better than DDR4 in HPCG, and Lulesh and HPCG for as many as 40% and 200%. For the selected compute intensive applications, the performance advantage of MCDRAM over DDR4 varies from 2% to 28%. We also observed that the cross-points, where MCDRAM starts outperforming DDR4, are around 8 to 16 threads.","PeriodicalId":292798,"journal":{"name":"Proceedings of the Seventh Workshop on Irregular Applications: Architectures and Algorithms","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130205041","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Synchronisation in the presence of noise and hardware performance variability is a key challenge that prevents applications from scaling to large problems and machines. Using asynchronous or semi-synchronous algorithms can help overcome this issue, but at the cost of reduced stability or convergence rate. In this paper we propose progressive load balancing to manage progress imbalance in asynchronous algorithms dynamically. In our technique the balancing is done over time, not instantaneously. Using Jacobi iterations as a test case, we show that, with CPU performance variability present, this approach leads to higher iteration rate and lower progress imbalance between parts of the solution space. We also show that under these conditions the balanced asynchronous method outperforms synchronous, semi-synchronous and totally asynchronous implementations in terms of time to solution.
{"title":"Progressive load balancing of asynchronous algorithms","authors":"Justs Zarins, Michèle Weiland","doi":"10.1145/3149704.3149765","DOIUrl":"https://doi.org/10.1145/3149704.3149765","url":null,"abstract":"Synchronisation in the presence of noise and hardware performance variability is a key challenge that prevents applications from scaling to large problems and machines. Using asynchronous or semi-synchronous algorithms can help overcome this issue, but at the cost of reduced stability or convergence rate. In this paper we propose progressive load balancing to manage progress imbalance in asynchronous algorithms dynamically. In our technique the balancing is done over time, not instantaneously. Using Jacobi iterations as a test case, we show that, with CPU performance variability present, this approach leads to higher iteration rate and lower progress imbalance between parts of the solution space. We also show that under these conditions the balanced asynchronous method outperforms synchronous, semi-synchronous and totally asynchronous implementations in terms of time to solution.","PeriodicalId":292798,"journal":{"name":"Proceedings of the Seventh Workshop on Irregular Applications: Architectures and Algorithms","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129782959","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Depth-First Search (DFS) is a pervasive algorithm, often used as a building block for topological sort, connectivity and planarity testing, among many other applications. We propose a novel work-efficient parallel algorithm for the DFS traversal of directed acyclic graph (DAG). The algorithm traverses the entire DAG in a BFS-like fashion no more than three times. As a result it finds the DFS pre-order (discovery) and post-order (finish time) as well as the parent relationship associated with every node in a DAG. We analyse the runtime and work complexity of this novel parallel algorithm. Also, we show that our algorithm is easy to implement and optimize for performance. In particular, we show that its CUDA implementation on the GPU outperforms sequential DFS on the CPU by up to 6x in our experiments.
深度优先搜索(deep - first Search, DFS)是一种普遍的算法，通常用作拓扑排序、连通性和平面性测试以及许多其他应用程序的构建块。针对有向无环图(DAG)的DFS遍历问题，提出了一种新的高效并行算法。该算法以类似bfs的方式遍历整个DAG不超过三次。因此，它可以找到DFS的预顺序(发现)和后顺序(完成时间)，以及与DAG中每个节点相关联的父关系。分析了该算法的运行时间和工作复杂度。此外，我们还证明了我们的算法易于实现和优化性能。特别是，在我们的实验中，我们表明它在GPU上的CUDA实现比CPU上的顺序DFS性能高出6倍。
{"title":"Parallel Depth-First Search for Directed Acyclic Graphs","authors":"M. Naumov, A. Vrielink, M. Garland","doi":"10.1145/3149704.3149764","DOIUrl":"https://doi.org/10.1145/3149704.3149764","url":null,"abstract":"Depth-First Search (DFS) is a pervasive algorithm, often used as a building block for topological sort, connectivity and planarity testing, among many other applications. We propose a novel work-efficient parallel algorithm for the DFS traversal of directed acyclic graph (DAG). The algorithm traverses the entire DAG in a BFS-like fashion no more than three times. As a result it finds the DFS pre-order (discovery) and post-order (finish time) as well as the parent relationship associated with every node in a DAG. We analyse the runtime and work complexity of this novel parallel algorithm. Also, we show that our algorithm is easy to implement and optimize for performance. In particular, we show that its CUDA implementation on the GPU outperforms sequential DFS on the CPU by up to 6x in our experiments.","PeriodicalId":292798,"journal":{"name":"Proceedings of the Seventh Workshop on Irregular Applications: Architectures and Algorithms","volume":"714 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126684101","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In particular, I will discuss the opportunities and challenges for irregular applications on quantum machines. Quantum machines face challenges of communication and control which will be affected by irregularity. Physical connectivity faces scaling challenges and hierarchical structures may be needed. Yet, underneath these similarities with classical machines, quantum computers involve non-local interactions that make them uniquely different.
{"title":"Quantum Computing and Irregular Applications","authors":"F. Chong","doi":"10.1145/3149704.3149773","DOIUrl":"https://doi.org/10.1145/3149704.3149773","url":null,"abstract":"In particular, I will discuss the opportunities and challenges for irregular applications on quantum machines. Quantum machines face challenges of communication and control which will be affected by irregularity. Physical connectivity faces scaling challenges and hierarchical structures may be needed. Yet, underneath these similarities with classical machines, quantum computers involve non-local interactions that make them uniquely different.","PeriodicalId":292798,"journal":{"name":"Proceedings of the Seventh Workshop on Irregular Applications: Architectures and Algorithms","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127900371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Given the increasing importance of efficient data intensive computing, we find that modern processor designs are not well suited to the irregular memory access patterns found in these algorithms. This research focuses on mapping the compiler's instruction cost scheduling logic to hardware managed concurrency controls in order to minimize pipeline stalls. In this manner, the hardware modules managing the low-latency thread concurrency can be directly understood by modern compilers. We introduce a thread context switching method that is managed directly via a set of hardware-based mechanisms that are coupled to the compiler instruction scheduler. As individual instructions from a thread execute, their respective cost is accumulated into a control register. Once the register reaches a pre-determined saturation point, the thread is forced to context switch. We evaluate the performance benefits of our approach using a series of 24 benchmarks that exhibit performance acceleration of up to 14.6X.
{"title":"Pressure-Driven Hardware Managed Thread Concurrency for Irregular Applications","authors":"John D. Leidel, Xi Wang, Yong Chen","doi":"10.1145/3149704.3149705","DOIUrl":"https://doi.org/10.1145/3149704.3149705","url":null,"abstract":"Given the increasing importance of efficient data intensive computing, we find that modern processor designs are not well suited to the irregular memory access patterns found in these algorithms. This research focuses on mapping the compiler's instruction cost scheduling logic to hardware managed concurrency controls in order to minimize pipeline stalls. In this manner, the hardware modules managing the low-latency thread concurrency can be directly understood by modern compilers. We introduce a thread context switching method that is managed directly via a set of hardware-based mechanisms that are coupled to the compiler instruction scheduler. As individual instructions from a thread execute, their respective cost is accumulated into a control register. Once the register reaches a pre-determined saturation point, the thread is forced to context switch. We evaluate the performance benefits of our approach using a series of 24 benchmarks that exhibit performance acceleration of up to 14.6X.","PeriodicalId":292798,"journal":{"name":"Proceedings of the Seventh Workshop on Irregular Applications: Architectures and Algorithms","volume":"184 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132036398","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Farzad Khorasani, Keval Vora, Rajiv Gupta, L. Bhuyan
Massive parallel processing power of GPUs has attracted researchers to develop iterative vertex-centric graph processing frameworks for GPUs. Enabling work-efficiency in these solutions, however, is not straightforward and comes at the cost of SIMD-inefficiency and load imbalance. This paper offers techniques that overcome these challenges when processing the graph on a GPU. For a SIMD-efficient kernel operation involving gathering of neighbors and performing reduction on them, we employ an effective task expansion strategy that avoids intra-warp thread underutilization. As recording vertex activeness requires additional data structures, to attenuate the graph storage overhead on limited GPU DRAM, we introduce vertex grouping as a technique that enables trade-off between memory consumption and the work efficiency in our solution. Our experiments show that these techniques provide up to 5.46x over the recently proposed WS-VR [4] framework over multiple algorithms and inputs.
{"title":"Enabling Work-Efficiency for High Performance Vertex-Centric Graph Analytics on GPUs","authors":"Farzad Khorasani, Keval Vora, Rajiv Gupta, L. Bhuyan","doi":"10.1145/3149704.3149762","DOIUrl":"https://doi.org/10.1145/3149704.3149762","url":null,"abstract":"Massive parallel processing power of GPUs has attracted researchers to develop iterative vertex-centric graph processing frameworks for GPUs. Enabling work-efficiency in these solutions, however, is not straightforward and comes at the cost of SIMD-inefficiency and load imbalance. This paper offers techniques that overcome these challenges when processing the graph on a GPU. For a SIMD-efficient kernel operation involving gathering of neighbors and performing reduction on them, we employ an effective task expansion strategy that avoids intra-warp thread underutilization. As recording vertex activeness requires additional data structures, to attenuate the graph storage overhead on limited GPU DRAM, we introduce vertex grouping as a technique that enables trade-off between memory consumption and the work efficiency in our solution. Our experiments show that these techniques provide up to 5.46x over the recently proposed WS-VR [4] framework over multiple algorithms and inputs.","PeriodicalId":292798,"journal":{"name":"Proceedings of the Seventh Workshop on Irregular Applications: Architectures and Algorithms","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121365864","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper we propose a load-balanced GPU kernel for computing the sparse matrix vector (SpMV) product. Making heavy use of the latest GPU programming features, we also enable satisfying performance for irregular and unbalanced matrices. In a performance comparison using 400 test matrices we reveal the new kernel being superior to the most popular SpMV implementations.
{"title":"Overcoming Load Imbalance for Irregular Sparse Matrices","authors":"Goran Flegar, H. Anzt","doi":"10.1145/3149704.3149767","DOIUrl":"https://doi.org/10.1145/3149704.3149767","url":null,"abstract":"In this paper we propose a load-balanced GPU kernel for computing the sparse matrix vector (SpMV) product. Making heavy use of the latest GPU programming features, we also enable satisfying performance for irregular and unbalanced matrices. In a performance comparison using 400 test matrices we reveal the new kernel being superior to the most popular SpMV implementations.","PeriodicalId":292798,"journal":{"name":"Proceedings of the Seventh Workshop on Irregular Applications: Architectures and Algorithms","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133365646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this short paper, we report the performance of multiple thread-parallel algorithms for spherical region queries on multicore architectures motivated by a challenging data analytics application in materials science. Performances of two tree-based algorithms and a naive algorithm are compared to identify the length scales at which these approaches perform optimally. The optimal algorithm is then used to scale the driving materials science application, which is shown to deliver over 17X speedup using 32 OpenMP threads on data sets containing many millions of atoms.
{"title":"Spherical Region Queries on Multicore Architectures","authors":"Hao Lu, S. Seal, Wei Guo, J. Poplawsky","doi":"10.1145/3149704.3149772","DOIUrl":"https://doi.org/10.1145/3149704.3149772","url":null,"abstract":"In this short paper, we report the performance of multiple thread-parallel algorithms for spherical region queries on multicore architectures motivated by a challenging data analytics application in materials science. Performances of two tree-based algorithms and a naive algorithm are compared to identify the length scales at which these approaches perform optimally. The optimal algorithm is then used to scale the driving materials science application, which is shown to deliver over 17X speedup using 32 OpenMP threads on data sets containing many millions of atoms.","PeriodicalId":292798,"journal":{"name":"Proceedings of the Seventh Workshop on Irregular Applications: Architectures and Algorithms","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129213385","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Modern supercomputers have millions of cores, each capable of executing one or more threads of program execution. In these computers the site of execution for program threads rarely, if ever, changes from the node in which they were born. This paper discusses the advantages that may accrue when thread states migrate freely from node to node, especially when migration is managed by hardware without requiring software intervention. Emphasis is on supporting the growing classes of algorithms where there is significant sparsity, irregularity, and lack of locality in the memory reference patterns. Evidence is drawn from reformulation of several kernels into a migrating thread context approximating that of an emerging architecture with such capabilities.
{"title":"A Case for Migrating Execution for Irregular Applications","authors":"P. Kogge, Shannon K. Kuntz","doi":"10.1145/3149704.3149770","DOIUrl":"https://doi.org/10.1145/3149704.3149770","url":null,"abstract":"Modern supercomputers have millions of cores, each capable of executing one or more threads of program execution. In these computers the site of execution for program threads rarely, if ever, changes from the node in which they were born. This paper discusses the advantages that may accrue when thread states migrate freely from node to node, especially when migration is managed by hardware without requiring software intervention. Emphasis is on supporting the growing classes of algorithms where there is significant sparsity, irregularity, and lack of locality in the memory reference patterns. Evidence is drawn from reformulation of several kernels into a migrating thread context approximating that of an emerging architecture with such capabilities.","PeriodicalId":292798,"journal":{"name":"Proceedings of the Seventh Workshop on Irregular Applications: Architectures and Algorithms","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-11-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128806956","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：