The Prony method for approximating signals comprising sinusoidal/exponential components is known through the pioneering work of Prony in his seminal dissertation in the year 1795. However, the Prony method saw the light of real world application only upon the advent of the computational era, which made feasible the extensive numerical intricacies and labor which the method demands inherently. The Adaptive LMS Filter which has been the most pervasive method for signal filtration and approximation since its inception in 1965 does not provide a consistently assured level of highly precise results as the extended experiment in this work proves. As a remedy this study improvises upon the Prony method by observing that a better (more precise) computational approximation can be obtained under the premise that adjustment can be made for computational error , in the autoregressive model setup in the initial step of the Prony computation itself. This adjustment is in proportion to the deviation of the coefficients in the same autoregressive model. The results obtained by this improvisation live up to the expectations of obtaining consistency and higher value in the precision of the output (recovered signal) approximations as shown in this current work and as compared with the results obtained using the Adaptive LMS Filter.
{"title":"A prony method variant which surpasses the Adaptive LMS filter in the output signal's representation of input","authors":"Parthasarathy Srinivasan","doi":"arxiv-2409.01272","DOIUrl":"https://doi.org/arxiv-2409.01272","url":null,"abstract":"The Prony method for approximating signals comprising sinusoidal/exponential\u0000components is known through the pioneering work of Prony in his seminal\u0000dissertation in the year 1795. However, the Prony method saw the light of real\u0000world application only upon the advent of the computational era, which made\u0000feasible the extensive numerical intricacies and labor which the method demands\u0000inherently. The Adaptive LMS Filter which has been the most pervasive method\u0000for signal filtration and approximation since its inception in 1965 does not\u0000provide a consistently assured level of highly precise results as the extended\u0000experiment in this work proves. As a remedy this study improvises upon the\u0000Prony method by observing that a better (more precise) computational\u0000approximation can be obtained under the premise that adjustment can be made for\u0000computational error , in the autoregressive model setup in the initial step of\u0000the Prony computation itself. This adjustment is in proportion to the deviation\u0000of the coefficients in the same autoregressive model. The results obtained by\u0000this improvisation live up to the expectations of obtaining consistency and\u0000higher value in the precision of the output (recovered signal) approximations\u0000as shown in this current work and as compared with the results obtained using\u0000the Adaptive LMS Filter.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"59 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142222702","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sibo Cheng, Jinyang Min, Che Liu, Rossella Arcucci
Data assimilation techniques are often confronted with challenges handling complex high dimensional physical systems, because high precision simulation in complex high dimensional physical systems is computationally expensive and the exact observation functions that can be applied in these systems are difficult to obtain. It prompts growing interest in integrating deep learning models within data assimilation workflows, but current software packages for data assimilation cannot handle deep learning models inside. This study presents a novel Python package seamlessly combining data assimilation with deep neural networks to serve as models for state transition and observation functions. The package, named TorchDA, implements Kalman Filter, Ensemble Kalman Filter (EnKF), 3D Variational (3DVar), and 4D Variational (4DVar) algorithms, allowing flexible algorithm selection based on application requirements. Comprehensive experiments conducted on the Lorenz 63 and a two-dimensional shallow water system demonstrate significantly enhanced performance over standalone model predictions without assimilation. The shallow water analysis validates data assimilation capabilities mapping between different physical quantity spaces in either full space or reduced order space. Overall, this innovative software package enables flexible integration of deep learning representations within data assimilation, conferring a versatile tool to tackle complex high dimensional dynamical systems across scientific domains.
{"title":"TorchDA: A Python package for performing data assimilation with deep learning forward and transformation functions","authors":"Sibo Cheng, Jinyang Min, Che Liu, Rossella Arcucci","doi":"arxiv-2409.00244","DOIUrl":"https://doi.org/arxiv-2409.00244","url":null,"abstract":"Data assimilation techniques are often confronted with challenges handling\u0000complex high dimensional physical systems, because high precision simulation in\u0000complex high dimensional physical systems is computationally expensive and the\u0000exact observation functions that can be applied in these systems are difficult\u0000to obtain. It prompts growing interest in integrating deep learning models\u0000within data assimilation workflows, but current software packages for data\u0000assimilation cannot handle deep learning models inside. This study presents a\u0000novel Python package seamlessly combining data assimilation with deep neural\u0000networks to serve as models for state transition and observation functions. The\u0000package, named TorchDA, implements Kalman Filter, Ensemble Kalman Filter\u0000(EnKF), 3D Variational (3DVar), and 4D Variational (4DVar) algorithms, allowing\u0000flexible algorithm selection based on application requirements. Comprehensive\u0000experiments conducted on the Lorenz 63 and a two-dimensional shallow water\u0000system demonstrate significantly enhanced performance over standalone model\u0000predictions without assimilation. The shallow water analysis validates data\u0000assimilation capabilities mapping between different physical quantity spaces in\u0000either full space or reduced order space. Overall, this innovative software\u0000package enables flexible integration of deep learning representations within\u0000data assimilation, conferring a versatile tool to tackle complex high\u0000dimensional dynamical systems across scientific domains.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"6 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142222701","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this study, we introduce HOBOTAN, a new solver designed for Higher Order Binary Optimization (HOBO). HOBOTAN supports both CPU and GPU, with the GPU version developed based on PyTorch, offering a fast and scalable system. This solver utilizes tensor networks to solve combinatorial optimization problems, employing a HOBO tensor that maps the problem and performs tensor contractions as needed. Additionally, by combining techniques such as batch processing for tensor optimization and binary-based integer encoding, we significantly enhance the efficiency of combinatorial optimization. In the future, the utilization of increased GPU numbers is expected to harness greater computational power, enabling efficient collaboration between multiple GPUs for high scalability. Moreover, HOBOTAN is designed within the framework of quantum computing, thus providing insights for future quantum computer applications. This paper details the design, implementation, performance evaluation, and scalability of HOBOTAN, demonstrating its effectiveness.
{"title":"HOBOTAN: Efficient Higher Order Binary Optimization Solver with Tensor Networks and PyTorch","authors":"Shoya Yasuda, Shunsuke Sotobayashi, Yuichiro Minato","doi":"arxiv-2407.19987","DOIUrl":"https://doi.org/arxiv-2407.19987","url":null,"abstract":"In this study, we introduce HOBOTAN, a new solver designed for Higher Order\u0000Binary Optimization (HOBO). HOBOTAN supports both CPU and GPU, with the GPU\u0000version developed based on PyTorch, offering a fast and scalable system. This\u0000solver utilizes tensor networks to solve combinatorial optimization problems,\u0000employing a HOBO tensor that maps the problem and performs tensor contractions\u0000as needed. Additionally, by combining techniques such as batch processing for\u0000tensor optimization and binary-based integer encoding, we significantly enhance\u0000the efficiency of combinatorial optimization. In the future, the utilization of\u0000increased GPU numbers is expected to harness greater computational power,\u0000enabling efficient collaboration between multiple GPUs for high scalability.\u0000Moreover, HOBOTAN is designed within the framework of quantum computing, thus\u0000providing insights for future quantum computer applications. This paper details\u0000the design, implementation, performance evaluation, and scalability of HOBOTAN,\u0000demonstrating its effectiveness.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"124 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141862913","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Stefano Chiaradonna, Petar Jevtic, Beckett Sterner
We present a Python package called Modular Petri Net Assembly Toolkit (MPAT) that empowers users to easily create large-scale, modular Petri Nets for various spatial configurations, including extensive spatial grids or those derived from shape files, augmented with heterogeneous information layers. Petri Nets are powerful discrete event system modeling tools in computational biology and engineering. However, their utility for automated construction of large-scale spatial models has been limited by gaps in existing modeling software packages. MPAT addresses this gap by supporting the development of modular Petri Net models with flexible spatial geometries.
我们介绍了一个名为模块化 Petri 网组装工具包（MPAT）的 Python 软件包，它能帮助用户轻松创建各种空间配置的大规模模块化 Petri 网，包括广泛的空间网格或从形状文件中生成的网格，并添加异构信息层。Petri 网是计算生物学和工程学领域强大的离散事件系统建模工具。Petri 网是计算生物学和工程学中强大的离散事件系统建模工具，但由于现有建模软件包的缺陷，其在自动构建大规模空间模型方面的实用性受到了限制。MPAT 支持开发具有灵活空间几何结构的模块化 Petri 网模型，从而弥补了这一不足。
{"title":"MPAT: Modular Petri Net Assembly Toolkit","authors":"Stefano Chiaradonna, Petar Jevtic, Beckett Sterner","doi":"arxiv-2407.10372","DOIUrl":"https://doi.org/arxiv-2407.10372","url":null,"abstract":"We present a Python package called Modular Petri Net Assembly Toolkit (MPAT)\u0000that empowers users to easily create large-scale, modular Petri Nets for\u0000various spatial configurations, including extensive spatial grids or those\u0000derived from shape files, augmented with heterogeneous information layers.\u0000Petri Nets are powerful discrete event system modeling tools in computational\u0000biology and engineering. However, their utility for automated construction of\u0000large-scale spatial models has been limited by gaps in existing modeling\u0000software packages. MPAT addresses this gap by supporting the development of\u0000modular Petri Net models with flexible spatial geometries.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"37 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141717812","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kacper Derlatka, Maciej Manna, Oleksii Bulenok, David Zwicker, Sylwester Arabas
The numba-mpi package offers access to the Message Passing Interface (MPI) routines from Python code that uses the Numba just-in-time (JIT) compiler. As a result, high-performance and multi-threaded Python code may utilize MPI communication facilities without leaving the JIT-compiled code blocks, which is not possible with the mpi4py package, a higher-level Python interface to MPI. For debugging purposes, numba-mpi retains full functionality of the code even if the JIT compilation is disabled. The numba-mpi API constitutes a thin wrapper around the C API of MPI and is built around Numpy arrays including handling of non-contiguous views over array slices. Project development is hosted at GitHub leveraging the mpi4py/setup-mpi workflow enabling continuous integration tests on Linux (MPICH, OpenMPI & Intel MPI), macOS (MPICH & OpenMPI) and Windows (MS MPI). The paper covers an overview of the package features, architecture and performance. As of v1.0, the following MPI routines are exposed and covered by unit tests: size/rank, [i]send/[i]recv, wait[all|any], test[all|any], allreduce, bcast, barrier, scatter/[all]gather & wtime. The package is implemented in pure Python and depends on numpy, numba and mpi4py (the latter used at initialization and as a source of utility routines only). The performance advantage of using numba-mpi compared to mpi4py is depicted with a simple example, with entirety of the code included in listings discussed in the text. Application of numba-mpi for handling domain decomposition in numerical solvers for partial differential equations is presented using two external packages that depend on numba-mpi: py-pde and PyMPDATA-MPI.
{"title":"Enabling MPI communication within Numba/LLVM JIT-compiled Python code using numba-mpi v1.0","authors":"Kacper Derlatka, Maciej Manna, Oleksii Bulenok, David Zwicker, Sylwester Arabas","doi":"arxiv-2407.13712","DOIUrl":"https://doi.org/arxiv-2407.13712","url":null,"abstract":"The numba-mpi package offers access to the Message Passing Interface (MPI)\u0000routines from Python code that uses the Numba just-in-time (JIT) compiler. As a\u0000result, high-performance and multi-threaded Python code may utilize MPI\u0000communication facilities without leaving the JIT-compiled code blocks, which is\u0000not possible with the mpi4py package, a higher-level Python interface to MPI.\u0000For debugging purposes, numba-mpi retains full functionality of the code even\u0000if the JIT compilation is disabled. The numba-mpi API constitutes a thin\u0000wrapper around the C API of MPI and is built around Numpy arrays including\u0000handling of non-contiguous views over array slices. Project development is\u0000hosted at GitHub leveraging the mpi4py/setup-mpi workflow enabling continuous\u0000integration tests on Linux (MPICH, OpenMPI & Intel MPI), macOS (MPICH &\u0000OpenMPI) and Windows (MS MPI). The paper covers an overview of the package\u0000features, architecture and performance. As of v1.0, the following MPI routines\u0000are exposed and covered by unit tests: size/rank, [i]send/[i]recv,\u0000wait[all|any], test[all|any], allreduce, bcast, barrier, scatter/[all]gather &\u0000wtime. The package is implemented in pure Python and depends on numpy, numba\u0000and mpi4py (the latter used at initialization and as a source of utility\u0000routines only). The performance advantage of using numba-mpi compared to mpi4py\u0000is depicted with a simple example, with entirety of the code included in\u0000listings discussed in the text. Application of numba-mpi for handling domain\u0000decomposition in numerical solvers for partial differential equations is\u0000presented using two external packages that depend on numba-mpi: py-pde and\u0000PyMPDATA-MPI.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"61 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141737352","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Formal verification (FV) has witnessed growing significance with current emerging program synthesis by the evolving large language models (LLMs). However, current formal verification mainly resorts to symbolic verifiers or hand-craft rules, resulting in limitations for extensive and flexible verification. On the other hand, formal languages for automated theorem proving, such as Isabelle, as another line of rigorous verification, are maintained with comprehensive rules and theorems. In this paper, we propose FVEL, an interactive Formal Verification Environment with LLMs. Specifically, FVEL transforms a given code to be verified into Isabelle, and then conducts verification via neural automated theorem proving with an LLM. The joined paradigm leverages the rigorous yet abundant formulated and organized rules in Isabelle and is also convenient for introducing and adjusting cutting-edge LLMs. To achieve this goal, we extract a large-scale FVELER3. The FVELER dataset includes code dependencies and verification processes that are formulated in Isabelle, containing 758 theories, 29,125 lemmas, and 200,646 proof steps in total with in-depth dependencies. We benchmark FVELER in the FVEL environment by first fine-tuning LLMs with FVELER and then evaluating them on Code2Inv and SV-COMP. The results show that FVEL with FVELER fine-tuned Llama3- 8B solves 17.39% (69 -> 81) more problems, and Mistral-7B 12% (75 -> 84) more problems in SV-COMP. And the proportion of proof errors is reduced. Project page: https://fveler.github.io/.
{"title":"FVEL: Interactive Formal Verification Environment with Large Language Models via Theorem Proving","authors":"Xiaohan Lin, Qingxing Cao, Yinya Huang, Haiming Wang, Jianqiao Lu, Zhengying Liu, Linqi Song, Xiaodan Liang","doi":"arxiv-2406.14408","DOIUrl":"https://doi.org/arxiv-2406.14408","url":null,"abstract":"Formal verification (FV) has witnessed growing significance with current\u0000emerging program synthesis by the evolving large language models (LLMs).\u0000However, current formal verification mainly resorts to symbolic verifiers or\u0000hand-craft rules, resulting in limitations for extensive and flexible\u0000verification. On the other hand, formal languages for automated theorem\u0000proving, such as Isabelle, as another line of rigorous verification, are\u0000maintained with comprehensive rules and theorems. In this paper, we propose\u0000FVEL, an interactive Formal Verification Environment with LLMs. Specifically,\u0000FVEL transforms a given code to be verified into Isabelle, and then conducts\u0000verification via neural automated theorem proving with an LLM. The joined\u0000paradigm leverages the rigorous yet abundant formulated and organized rules in\u0000Isabelle and is also convenient for introducing and adjusting cutting-edge\u0000LLMs. To achieve this goal, we extract a large-scale FVELER3. The FVELER\u0000dataset includes code dependencies and verification processes that are\u0000formulated in Isabelle, containing 758 theories, 29,125 lemmas, and 200,646\u0000proof steps in total with in-depth dependencies. We benchmark FVELER in the\u0000FVEL environment by first fine-tuning LLMs with FVELER and then evaluating them\u0000on Code2Inv and SV-COMP. The results show that FVEL with FVELER fine-tuned\u0000Llama3- 8B solves 17.39% (69 -> 81) more problems, and Mistral-7B 12% (75 ->\u000084) more problems in SV-COMP. And the proportion of proof errors is reduced.\u0000Project page: https://fveler.github.io/.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"76 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141529288","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
OpenCAEPoro is a parallel numerical simulation software developed in C++ for simulating multiphase and multicomponent flows in porous media. The software utilizes a set of general-purpose compositional model equations, enabling it to handle a diverse range of fluid dynamics, including the black oil model, compositional model, and thermal recovery models. OpenCAEPoro establishes a unified solving framework that integrates many widely used methods, such as IMPEC, FIM, and AIM. This framework allows dynamic collaboration between different methods. Specifically, based on this framework, we have developed an adaptively coupled domain decomposition method, which can provide initial solutions for global methods to accelerate the simulation. The reliability of OpenCAEPoro has been validated through benchmark testing with the SPE comparative solution project. Furthermore, its robust parallel efficiency has been tested in distributed parallel environments, demonstrating its suitability for large-scale simulation problems.
{"title":"OpenCAEPoro: A Parallel Simulation Framework for Multiphase and Multicomponent Porous Media Flows","authors":"Shizhe Li, Chen-Song Zhang","doi":"arxiv-2406.10862","DOIUrl":"https://doi.org/arxiv-2406.10862","url":null,"abstract":"OpenCAEPoro is a parallel numerical simulation software developed in C++ for\u0000simulating multiphase and multicomponent flows in porous media. The software\u0000utilizes a set of general-purpose compositional model equations, enabling it to\u0000handle a diverse range of fluid dynamics, including the black oil model,\u0000compositional model, and thermal recovery models. OpenCAEPoro establishes a\u0000unified solving framework that integrates many widely used methods, such as\u0000IMPEC, FIM, and AIM. This framework allows dynamic collaboration between\u0000different methods. Specifically, based on this framework, we have developed an\u0000adaptively coupled domain decomposition method, which can provide initial\u0000solutions for global methods to accelerate the simulation. The reliability of\u0000OpenCAEPoro has been validated through benchmark testing with the SPE\u0000comparative solution project. Furthermore, its robust parallel efficiency has\u0000been tested in distributed parallel environments, demonstrating its suitability\u0000for large-scale simulation problems.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"14 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141506254","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Symmetric and sparse tensors arise naturally in many domains including linear algebra, statistics, physics, chemistry, and graph theory. Symmetric tensors are equal to their transposes, so in the $n$-dimensional case we can save up to a factor of $n!$ by avoiding redundant operations. Sparse tensors, on the other hand, are mostly zero, and we can save asymptotically by processing only nonzeros. Unfortunately, specializing for both symmetry and sparsity at the same time is uniquely challenging. Optimizing for symmetry requires consideration of $n!$ transpositions of a triangular kernel, which can be complex and error prone. Considering multiple transposed iteration orders and triangular loop bounds also complicates iteration through intricate sparse tensor formats. Additionally, since each combination of symmetry and sparse tensor formats requires a specialized implementation, this leads to a combinatorial number of cases. A compiler is needed, but existing compilers cannot take advantage of both symmetry and sparsity within the same kernel. In this paper, we describe the first compiler which can automatically generate symmetry-aware code for sparse or structured tensor kernels. We introduce a taxonomy for symmetry in tensor kernels, and show how to target each kind of symmetry. Our implementation demonstrates significant speedups ranging from 1.36x for SSYMV to 30.4x for a 5-dimensional MTTKRP over the non-symmetric state of the art.
{"title":"SySTeC: A Symmetric Sparse Tensor Compiler","authors":"Radha Patel, Willow Ahrens, Saman Amarasinghe","doi":"arxiv-2406.09266","DOIUrl":"https://doi.org/arxiv-2406.09266","url":null,"abstract":"Symmetric and sparse tensors arise naturally in many domains including linear\u0000algebra, statistics, physics, chemistry, and graph theory. Symmetric tensors\u0000are equal to their transposes, so in the $n$-dimensional case we can save up to\u0000a factor of $n!$ by avoiding redundant operations. Sparse tensors, on the other\u0000hand, are mostly zero, and we can save asymptotically by processing only\u0000nonzeros. Unfortunately, specializing for both symmetry and sparsity at the\u0000same time is uniquely challenging. Optimizing for symmetry requires\u0000consideration of $n!$ transpositions of a triangular kernel, which can be\u0000complex and error prone. Considering multiple transposed iteration orders and\u0000triangular loop bounds also complicates iteration through intricate sparse\u0000tensor formats. Additionally, since each combination of symmetry and sparse\u0000tensor formats requires a specialized implementation, this leads to a\u0000combinatorial number of cases. A compiler is needed, but existing compilers\u0000cannot take advantage of both symmetry and sparsity within the same kernel. In\u0000this paper, we describe the first compiler which can automatically generate\u0000symmetry-aware code for sparse or structured tensor kernels. We introduce a\u0000taxonomy for symmetry in tensor kernels, and show how to target each kind of\u0000symmetry. Our implementation demonstrates significant speedups ranging from\u00001.36x for SSYMV to 30.4x for a 5-dimensional MTTKRP over the non-symmetric\u0000state of the art.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"9 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141506256","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Richard Tran Mills, Mark Adams, Satish Balay, Jed Brown, Jacob Faibussowitsch, Toby Isaac, Matthew Knepley, Todd Munson, Hansol Suh, Stefano Zampini, Hong Zhang, Junchao Zhang
The Portable Extensible Toolkit for Scientific Computation (PETSc) library provides scalable solvers for nonlinear time-dependent differential and algebraic equations and for numerical optimization via the Toolkit for Advanced Optimization (TAO). PETSc is used in dozens of scientific fields and is an important building block for many simulation codes. During the U.S. Department of Energy's Exascale Computing Project, the PETSc team has made substantial efforts to enable efficient utilization of the massive fine-grain parallelism present within exascale compute nodes and to enable performance portability across exascale architectures. We recap some of the challenges that designers of numerical libraries face in such an endeavor, and then discuss the many developments we have made, which include the addition of new GPU backends, features supporting efficient on-device matrix assembly, better support for asynchronicity and GPU kernel concurrency, and new communication infrastructure. We evaluate the performance of these developments on some pre-exascale systems as well the early exascale systems Frontier and Aurora, using compute kernel, communication layer, solver, and mini-application benchmark studies, and then close with a few observations drawn from our experiences on the tension between portable performance and other goals of numerical libraries.
{"title":"PETSc/TAO Developments for Early Exascale Systems","authors":"Richard Tran Mills, Mark Adams, Satish Balay, Jed Brown, Jacob Faibussowitsch, Toby Isaac, Matthew Knepley, Todd Munson, Hansol Suh, Stefano Zampini, Hong Zhang, Junchao Zhang","doi":"arxiv-2406.08646","DOIUrl":"https://doi.org/arxiv-2406.08646","url":null,"abstract":"The Portable Extensible Toolkit for Scientific Computation (PETSc) library\u0000provides scalable solvers for nonlinear time-dependent differential and\u0000algebraic equations and for numerical optimization via the Toolkit for Advanced\u0000Optimization (TAO). PETSc is used in dozens of scientific fields and is an\u0000important building block for many simulation codes. During the U.S. Department\u0000of Energy's Exascale Computing Project, the PETSc team has made substantial\u0000efforts to enable efficient utilization of the massive fine-grain parallelism\u0000present within exascale compute nodes and to enable performance portability\u0000across exascale architectures. We recap some of the challenges that designers\u0000of numerical libraries face in such an endeavor, and then discuss the many\u0000developments we have made, which include the addition of new GPU backends,\u0000features supporting efficient on-device matrix assembly, better support for\u0000asynchronicity and GPU kernel concurrency, and new communication\u0000infrastructure. We evaluate the performance of these developments on some\u0000pre-exascale systems as well the early exascale systems Frontier and Aurora,\u0000using compute kernel, communication layer, solver, and mini-application\u0000benchmark studies, and then close with a few observations drawn from our\u0000experiences on the tension between portable performance and other goals of\u0000numerical libraries.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"168 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141506257","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pawel K. Radtke, Cristian G. Barrera-Hinojosa, Mladen Ivkovic, Tobias Weinzierl
The C++ programming language and its cousins lean towards a memory-inefficient storage of structs: The compiler inserts helper bits into the struct such that individual attributes align with bytes, and it adds additional bytes aligning attributes with cache lines, while it is not able to exploit knowledge about the range of integers, enums or bitsets to bring the memory footprint down. Furthermore, the language provides neither support for data exchange via MPI nor for arbitrary floating-point precision formats. If developers need to have a low memory footprint and MPI datatypes over structs which exchange only minimal data, they have to manipulate the data and to write MPI datatypes manually. We propose a C++ language extension based upon C++ attributes through which developers can guide the compiler what memory arrangements would be beneficial: Can multiple booleans be squeezed into one bit field, do floats hold fewer significant bits than in the IEEE standard, or does the code require a user-defined MPI datatype for certain subsets of attributes? The extension offers the opportunity to fall back to normal alignment and padding rules via plain C++ assignments, no dependencies upon external libraries are introduced, and the resulting code remains standard C++. Our work implements the language annotations within LLVM and demonstrates their potential impact, both upon the runtime and the memory footprint, through smoothed particle hydrodynamics (SPH) benchmarks. They uncover the potential gains in terms of performance and development productivity.
C++ 编程语言及其同类语言倾向于低内存效率的结构体存储：编译器会在结构体中插入辅助位，使单个属性与字节对齐，并添加额外的字节使属性与缓存行对齐，而无法利用整数、枚举或比特集的范围知识来减少内存占用。此外，该语言既不支持通过 MPI 进行数据交换，也不支持任意浮点精度格式。如果开发人员需要在只交换极少量数据的结构体上使用低内存占用和 MPI 数据类型，就必须手动操作数据和编写 MPI 数据类型。我们提出了一种基于 C++ 属性的 C++ 语言扩展，开发人员可以通过它指导编译器如何安排内存：是否可以将多个布尔值挤入一个比特字段，浮点数是否比 IEEE 标准中的有效位数更少，代码是否需要为某些属性子集提供用户定义的 MPI 数据类型？我们的工作在 LLVM 中实现了语言注释，并通过平滑粒子流体力学（SPH）基准测试证明了语言注释对运行时间和内存占用的潜在影响。我们的工作在 LLVM 中实现了语言注解，并通过平滑粒子流体力学（SPH）基准测试证明了其对运行时间和内存占用的潜在影响。
{"title":"An extension of C++ with memory-centric specifications for HPC to reduce memory footprints and streamline MPI development","authors":"Pawel K. Radtke, Cristian G. Barrera-Hinojosa, Mladen Ivkovic, Tobias Weinzierl","doi":"arxiv-2406.06095","DOIUrl":"https://doi.org/arxiv-2406.06095","url":null,"abstract":"The C++ programming language and its cousins lean towards a\u0000memory-inefficient storage of structs: The compiler inserts helper bits into\u0000the struct such that individual attributes align with bytes, and it adds\u0000additional bytes aligning attributes with cache lines, while it is not able to\u0000exploit knowledge about the range of integers, enums or bitsets to bring the\u0000memory footprint down. Furthermore, the language provides neither support for\u0000data exchange via MPI nor for arbitrary floating-point precision formats. If\u0000developers need to have a low memory footprint and MPI datatypes over structs\u0000which exchange only minimal data, they have to manipulate the data and to write\u0000MPI datatypes manually. We propose a C++ language extension based upon C++\u0000attributes through which developers can guide the compiler what memory\u0000arrangements would be beneficial: Can multiple booleans be squeezed into one\u0000bit field, do floats hold fewer significant bits than in the IEEE standard, or\u0000does the code require a user-defined MPI datatype for certain subsets of\u0000attributes? The extension offers the opportunity to fall back to normal\u0000alignment and padding rules via plain C++ assignments, no dependencies upon\u0000external libraries are introduced, and the resulting code remains standard C++.\u0000Our work implements the language annotations within LLVM and demonstrates their\u0000potential impact, both upon the runtime and the memory footprint, through\u0000smoothed particle hydrodynamics (SPH) benchmarks. They uncover the potential\u0000gains in terms of performance and development productivity.","PeriodicalId":501256,"journal":{"name":"arXiv - CS - Mathematical Software","volume":"233 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141506259","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：