Huan He, Ziyuan Tang, Shifan Zhao, Yousef Saad, Yuanzhe Xi
SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 712-743, March 2024. Abstract. This paper develops a new class of nonlinear acceleration algorithms based on extending conjugate residual-type procedures from linear to nonlinear equations. The main algorithm has strong similarities with Anderson acceleration as well as with inexact Newton methods—depending on which variant is implemented. We prove theoretically and verify experimentally, on a variety of problems from simulation experiments to deep learning applications, that our method is a powerful accelerated iterative algorithm. The code is available at https://github.com/Data-driven-numerical-methods/Nonlinear-Truncated-Conjugate-Residual.
{"title":"nlTGCR: A Class of Nonlinear Acceleration Procedures Based on Conjugate Residuals","authors":"Huan He, Ziyuan Tang, Shifan Zhao, Yousef Saad, Yuanzhe Xi","doi":"10.1137/23m1576360","DOIUrl":"https://doi.org/10.1137/23m1576360","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 712-743, March 2024. <br/> Abstract. This paper develops a new class of nonlinear acceleration algorithms based on extending conjugate residual-type procedures from linear to nonlinear equations. The main algorithm has strong similarities with Anderson acceleration as well as with inexact Newton methods—depending on which variant is implemented. We prove theoretically and verify experimentally, on a variety of problems from simulation experiments to deep learning applications, that our method is a powerful accelerated iterative algorithm. The code is available at https://github.com/Data-driven-numerical-methods/Nonlinear-Truncated-Conjugate-Residual.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-02-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140006558","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zachary M. Boyd, Nicolas Fraiman, Jeremy L. Marzuola, Peter J. Mucha, Braxton Osting
SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 685-711, March 2024. Abstract. We provide a rearrangement based algorithm for detection of subgraphs of k vertices with long escape times for directed or undirected networks that is not combinatorially complex to compute. Complementing other notions of densest subgraphs and graph cuts, our method is based on the mean hitting time required for a random walker to leave a designated set and hit the complement. We provide a new relaxation of this notion of hitting time on a given subgraph and use that relaxation to construct a subgraph detection algorithm that can be computed easily and a generalization to K-partitioning schemes. Using a modification of the subgraph detector on each component, we propose a graph partitioner that identifies regions where random walks live for comparably large times. Importantly, our method implicitly respects the directed nature of the data for directed graphs while also being applicable to undirected graphs. We apply the partitioning method for community detection to a large class of models and real-world data sets.
SIAM 矩阵分析与应用期刊》，第 45 卷，第 1 期，第 685-711 页，2024 年 3 月。 摘要。我们提供了一种基于重排的算法，用于检测有向或无向网络中逃逸时间较长的 k 个顶点的子图，其计算并不复杂。作为对其他最密子图和图切割概念的补充，我们的方法基于随机漫步者离开指定集合并命中补集所需的平均命中时间。我们对给定子图上的命中时间这一概念进行了新的松弛，并利用这一松弛构建了一种可以轻松计算的子图检测算法，并将其推广到 K 分区方案中。利用对每个组件上的子图检测器的修改，我们提出了一种图分割器，它能识别随机游走存活时间相当大的区域。重要的是，我们的方法隐含地尊重了有向图数据的有向性，同时也适用于无向图。我们将群落检测的分区方法应用于一大类模型和真实世界的数据集。
{"title":"An Escape Time Formulation for Subgraph Detection and Partitioning of Directed Graphs","authors":"Zachary M. Boyd, Nicolas Fraiman, Jeremy L. Marzuola, Peter J. Mucha, Braxton Osting","doi":"10.1137/23m1553790","DOIUrl":"https://doi.org/10.1137/23m1553790","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 685-711, March 2024. <br/> Abstract. We provide a rearrangement based algorithm for detection of subgraphs of k vertices with long escape times for directed or undirected networks that is not combinatorially complex to compute. Complementing other notions of densest subgraphs and graph cuts, our method is based on the mean hitting time required for a random walker to leave a designated set and hit the complement. We provide a new relaxation of this notion of hitting time on a given subgraph and use that relaxation to construct a subgraph detection algorithm that can be computed easily and a generalization to K-partitioning schemes. Using a modification of the subgraph detector on each component, we propose a graph partitioner that identifies regions where random walks live for comparably large times. Importantly, our method implicitly respects the directed nature of the data for directed graphs while also being applicable to undirected graphs. We apply the partitioning method for community detection to a large class of models and real-world data sets.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139968531","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 661-684, March 2024. Abstract. Given a family of nearly commuting symmetric matrices, we consider the task of computing an orthogonal matrix that nearly diagonalizes every matrix in the family. In this paper, we propose and analyze randomized joint diagonalization (RJD) for performing this task. RJD applies a standard eigenvalue solver to random linear combinations of the matrices. Unlike existing optimization-based methods, RJD is simple to implement and leverages existing high-quality linear algebra software packages. Our main novel contribution is to prove robust recovery: Given a family that is [math]-near to a commuting family, RJD jointly diagonalizes this family, with high probability, up to an error of norm [math]. We also discuss how the algorithm can be further improved by deflation techniques and demonstrate its state-of-the-art performance by numerical experiments with synthetic and real-world data.
{"title":"Randomized Joint Diagonalization of Symmetric Matrices","authors":"Haoze He, Daniel Kressner","doi":"10.1137/22m1541265","DOIUrl":"https://doi.org/10.1137/22m1541265","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 661-684, March 2024. <br/> Abstract. Given a family of nearly commuting symmetric matrices, we consider the task of computing an orthogonal matrix that nearly diagonalizes every matrix in the family. In this paper, we propose and analyze randomized joint diagonalization (RJD) for performing this task. RJD applies a standard eigenvalue solver to random linear combinations of the matrices. Unlike existing optimization-based methods, RJD is simple to implement and leverages existing high-quality linear algebra software packages. Our main novel contribution is to prove robust recovery: Given a family that is [math]-near to a commuting family, RJD jointly diagonalizes this family, with high probability, up to an error of norm [math]. We also discuss how the algorithm can be further improved by deflation techniques and demonstrate its state-of-the-art performance by numerical experiments with synthetic and real-world data.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139969714","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 634-660, March 2024. Abstract. Matrix factorizations in dual number algebra, a hypercomplex number system, have been applied to kinematics, spatial mechanisms, and other fields recently. We develop an approach to identify spatiotemporal patterns in the brain such as traveling waves using the singular value decomposition (SVD) of dual matrices in this paper. Theoretically, we propose the compact dual singular value decomposition (CDSVD) of dual complex matrices with explicit expressions as well as a necessary and sufficient condition for its existence. Furthermore, based on the CDSVD, we report on the optimal solution to the best rank-[math] approximation under a newly defined quasi-metric in the dual complex number system. The CDSVD is also related to the dual Moore–Penrose generalized inverse. Numerically, comparisons with other available algorithms are conducted, which indicate less computational costs of our proposed CDSVD. In addition, the infinitesimal part of the CDSVD can identify the true rank of the original matrix from the noise-added matrix, but the classical SVD cannot. Next, we employ experiments on simulated time-series data and a road monitoring video to demonstrate the beneficial effect of the infinitesimal parts of dual matrices in spatiotemporal pattern identification. Finally, we apply this approach to the large-scale brain functional magnetic resonance imaging data, identify three kinds of traveling waves, and further validate the consistency between our analytical results and the current knowledge of cerebral cortex function.
{"title":"Singular Value Decomposition of Dual Matrices and its Application to Traveling Wave Identification in the Brain","authors":"Tong Wei, Weiyang Ding, Yimin Wei","doi":"10.1137/23m1556642","DOIUrl":"https://doi.org/10.1137/23m1556642","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 634-660, March 2024. <br/> Abstract. Matrix factorizations in dual number algebra, a hypercomplex number system, have been applied to kinematics, spatial mechanisms, and other fields recently. We develop an approach to identify spatiotemporal patterns in the brain such as traveling waves using the singular value decomposition (SVD) of dual matrices in this paper. Theoretically, we propose the compact dual singular value decomposition (CDSVD) of dual complex matrices with explicit expressions as well as a necessary and sufficient condition for its existence. Furthermore, based on the CDSVD, we report on the optimal solution to the best rank-[math] approximation under a newly defined quasi-metric in the dual complex number system. The CDSVD is also related to the dual Moore–Penrose generalized inverse. Numerically, comparisons with other available algorithms are conducted, which indicate less computational costs of our proposed CDSVD. In addition, the infinitesimal part of the CDSVD can identify the true rank of the original matrix from the noise-added matrix, but the classical SVD cannot. Next, we employ experiments on simulated time-series data and a road monitoring video to demonstrate the beneficial effect of the infinitesimal parts of dual matrices in spatiotemporal pattern identification. Finally, we apply this approach to the large-scale brain functional magnetic resonance imaging data, identify three kinds of traveling waves, and further validate the consistency between our analytical results and the current knowledge of cerebral cortex function.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-02-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139756211","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Alice Cortinovis, Daniel Kressner, Yuji Nakatsukasa
SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 619-633, March 2024. Abstract. This work is concerned with the computation of the action of a matrix function f(A), such as the matrix exponential or the matrix square root, on a vector b. For a general matrix A, this can be done by computing the compression of A onto a suitable Krylov subspace. Such compression is usually computed by forming an orthonormal basis of the Krylov subspace using the Arnoldi method. In this work, we propose to compute (nonorthonormal) bases in a faster way and to use a fast randomized algorithm for least-squares problems to compute the compression of A onto the Krylov subspace. We present some numerical examples which show that our algorithms can be faster than the standard Arnoldi method while achieving comparable accuracy.
SIAM 矩阵分析与应用期刊》，第 45 卷，第 1 期，第 619-633 页，2024 年 3 月。 摘要。这项工作涉及矩阵函数 f(A) 对向量 b 的作用的计算，例如矩阵指数或矩阵平方根。对于一般矩阵 A，可以通过计算 A 对合适的 Krylov 子空间的压缩来实现。这种压缩通常是通过使用 Arnoldi 方法形成 Krylov 子空间的正交基来计算的。在这项工作中，我们建议以更快的方式计算（非正态）基，并使用最小二乘问题的快速随机算法来计算 A 到 Krylov 子空间的压缩。我们给出了一些数值示例，表明我们的算法比标准阿诺德方法更快，同时精度相当。
{"title":"Speeding Up Krylov Subspace Methods for Computing [math] via Randomization","authors":"Alice Cortinovis, Daniel Kressner, Yuji Nakatsukasa","doi":"10.1137/22m1543458","DOIUrl":"https://doi.org/10.1137/22m1543458","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 619-633, March 2024. <br/> Abstract. This work is concerned with the computation of the action of a matrix function f(A), such as the matrix exponential or the matrix square root, on a vector b. For a general matrix A, this can be done by computing the compression of A onto a suitable Krylov subspace. Such compression is usually computed by forming an orthonormal basis of the Krylov subspace using the Arnoldi method. In this work, we propose to compute (nonorthonormal) bases in a faster way and to use a fast randomized algorithm for least-squares problems to compute the compression of A onto the Krylov subspace. We present some numerical examples which show that our algorithms can be faster than the standard Arnoldi method while achieving comparable accuracy.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-02-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139756206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 601-618, March 2024. Abstract. In prioritization schemes, based on pairwise comparisons, such as the analytical hierarchy process, it is important to extract a cardinal ranking vector from a reciprocal matrix that is unlikely to be consistent. It is natural to choose such a vector only from efficient ones. Recently, a method to generate inductively all efficient vectors for any reciprocal matrix has been discovered. Here we focus on the study of efficient vectors for a reciprocal matrix that is a block perturbation of a consistent matrix in the sense that it is obtained from a consistent matrix by modifying entries only in a proper principal submatrix. We determine an explicit class of efficient vectors for such matrices. Based on this, we give a description of all the efficient vectors in the 3-by-3 block perturbed case. In addition, we give sufficient conditions for the right Perron eigenvector of such matrices to be efficient and provide examples in which efficiency does not occur. Also, we consider a certain type of constant block perturbed consistent matrices, for which we may construct a class of efficient vectors, and demonstrate the efficiency of the Perron eigenvector. Appropriate examples are provided throughout.
{"title":"Efficient Vectors for Block Perturbed Consistent Matrices","authors":"Susana Furtado, Charles Johnson","doi":"10.1137/23m1580310","DOIUrl":"https://doi.org/10.1137/23m1580310","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 601-618, March 2024. <br/> Abstract. In prioritization schemes, based on pairwise comparisons, such as the analytical hierarchy process, it is important to extract a cardinal ranking vector from a reciprocal matrix that is unlikely to be consistent. It is natural to choose such a vector only from efficient ones. Recently, a method to generate inductively all efficient vectors for any reciprocal matrix has been discovered. Here we focus on the study of efficient vectors for a reciprocal matrix that is a block perturbation of a consistent matrix in the sense that it is obtained from a consistent matrix by modifying entries only in a proper principal submatrix. We determine an explicit class of efficient vectors for such matrices. Based on this, we give a description of all the efficient vectors in the 3-by-3 block perturbed case. In addition, we give sufficient conditions for the right Perron eigenvector of such matrices to be efficient and provide examples in which efficiency does not occur. Also, we consider a certain type of constant block perturbed consistent matrices, for which we may construct a class of efficient vectors, and demonstrate the efficiency of the Perron eigenvector. Appropriate examples are provided throughout.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139756194","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 553-584, March 2024. Abstract. It is a classical task in perturbation analysis to find norm bounds on the effect of a perturbation [math] of a stochastic matrix [math] to its stationary distribution, i.e., to the unique normalized left Perron eigenvector. A common assumption is to consider [math] to be given and to find bounds on its impact, but in this paper, we rather focus on an inverse optimization problem called the target stationary distribution problem (TSDP). The starting point is a target stationary distribution, and we search for a perturbation [math] of the minimum norm such that [math] remains stochastic and has the desired target stationary distribution. It is shown that TSDP has relevant applications in the design of, for example, road networks, social networks, hyperlink networks, and queuing systems. The key to our approach is that we work with rank-1 perturbations. Building on those results for rank-1 perturbations, we provide heuristics for the TSDP that construct arbitrary rank perturbations as sums of appropriately constructed rank-1 perturbations.
{"title":"Perturbation and Inverse Problems of Stochastic Matrices","authors":"Joost Berkhout, Bernd Heidergott, Paul Van Dooren","doi":"10.1137/22m1489162","DOIUrl":"https://doi.org/10.1137/22m1489162","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 553-584, March 2024. <br/> Abstract. It is a classical task in perturbation analysis to find norm bounds on the effect of a perturbation [math] of a stochastic matrix [math] to its stationary distribution, i.e., to the unique normalized left Perron eigenvector. A common assumption is to consider [math] to be given and to find bounds on its impact, but in this paper, we rather focus on an inverse optimization problem called the target stationary distribution problem (TSDP). The starting point is a target stationary distribution, and we search for a perturbation [math] of the minimum norm such that [math] remains stochastic and has the desired target stationary distribution. It is shown that TSDP has relevant applications in the design of, for example, road networks, social networks, hyperlink networks, and queuing systems. The key to our approach is that we work with rank-1 perturbations. Building on those results for rank-1 perturbations, we provide heuristics for the TSDP that construct arbitrary rank perturbations as sums of appropriately constructed rank-1 perturbations.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139756461","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 585-600, March 2024. Abstract. It is widely believed that typical finite families of [math] matrices admit finite products that attain the joint spectral radius. This conjecture is supported by computational experiments and it naturally leads to the following question: are these spectrum maximizing products typically unique, up to cyclic permutations and powers? We answer this question negatively. As discovered by Horowitz around fifty years ago, there are products of matrices that always have the same spectral radius despite not being cyclic permutations of one another. We show that the simplest Horowitz products can be spectrum maximizing in a robust way; more precisely, we exhibit a small but nonempty open subset of pairs of [math] matrices [math] for which the products [math] and [math] are both spectrum maximizing.
{"title":"Spectrum Maximizing Products Are Not Generically Unique","authors":"Jairo Bochi, Piotr Laskawiec","doi":"10.1137/23m1550621","DOIUrl":"https://doi.org/10.1137/23m1550621","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 585-600, March 2024. <br/> Abstract. It is widely believed that typical finite families of [math] matrices admit finite products that attain the joint spectral radius. This conjecture is supported by computational experiments and it naturally leads to the following question: are these spectrum maximizing products typically unique, up to cyclic permutations and powers? We answer this question negatively. As discovered by Horowitz around fifty years ago, there are products of matrices that always have the same spectral radius despite not being cyclic permutations of one another. We show that the simplest Horowitz products can be spectrum maximizing in a robust way; more precisely, we exhibit a small but nonempty open subset of pairs of [math] matrices [math] for which the products [math] and [math] are both spectrum maximizing.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139756204","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Patrick Amestoy, Alfredo Buttari, Nicholas J. Higham, Jean-Yves L’Excellent, Theo Mary, Bastien Vieublé
SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 529-552, March 2024. Abstract. GMRES-based iterative refinement in three precisions (GMRES-IR3), proposed by Carson and Higham in 2018, uses a low precision LU factorization to accelerate the solution of a linear system without compromising numerical stability or robustness. GMRES-IR3 solves the update equation of iterative refinement using GMRES preconditioned by the LU factors, where all operations within GMRES are carried out in the working precision [math], except for the matrix–vector products and the application of the preconditioner, which require the use of extra precision [math]. The use of extra precision can be expensive, and is especially unattractive if it is not available in hardware; for this reason, existing implementations have not used extra precision, despite the absence of an error analysis for this approach. In this article, we propose to relax the requirements on the precisions used within GMRES, allowing the use of arbitrary precisions [math] for applying the preconditioned matrix–vector product and [math] for the rest of the operations. We obtain the five-precision GMRES-based iterative refinement (GMRES-IR5) algorithm which has the potential to solve relatively badly conditioned problems in less time and memory than GMRES-IR3. We develop a rounding error analysis that generalizes that of GMRES-IR3, obtaining conditions under which the forward and backward errors converge to their limiting values. Our analysis makes use of a new result on the backward stability of MGS-GMRES in two precisions. On hardware where three or more arithmetics are available, which is becoming very common, the number of possible combinations of precisions in GMRES-IR5 is extremely large. We provide an analysis of our theoretical results that identifies a relatively small subset of relevant combinations. By choosing from within this subset one can achieve different levels of tradeoff between cost and robustness, which allows for a finer choice of precisions depending on the problem difficulty and the available hardware. We carry out numerical experiments on random dense and SuiteSparse matrices to validate our theoretical analysis and discuss the complexity of GMRES-IR5.
{"title":"Five-Precision GMRES-Based Iterative Refinement","authors":"Patrick Amestoy, Alfredo Buttari, Nicholas J. Higham, Jean-Yves L’Excellent, Theo Mary, Bastien Vieublé","doi":"10.1137/23m1549079","DOIUrl":"https://doi.org/10.1137/23m1549079","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 529-552, March 2024. <br/> Abstract. GMRES-based iterative refinement in three precisions (GMRES-IR3), proposed by Carson and Higham in 2018, uses a low precision LU factorization to accelerate the solution of a linear system without compromising numerical stability or robustness. GMRES-IR3 solves the update equation of iterative refinement using GMRES preconditioned by the LU factors, where all operations within GMRES are carried out in the working precision [math], except for the matrix–vector products and the application of the preconditioner, which require the use of extra precision [math]. The use of extra precision can be expensive, and is especially unattractive if it is not available in hardware; for this reason, existing implementations have not used extra precision, despite the absence of an error analysis for this approach. In this article, we propose to relax the requirements on the precisions used within GMRES, allowing the use of arbitrary precisions [math] for applying the preconditioned matrix–vector product and [math] for the rest of the operations. We obtain the five-precision GMRES-based iterative refinement (GMRES-IR5) algorithm which has the potential to solve relatively badly conditioned problems in less time and memory than GMRES-IR3. We develop a rounding error analysis that generalizes that of GMRES-IR3, obtaining conditions under which the forward and backward errors converge to their limiting values. Our analysis makes use of a new result on the backward stability of MGS-GMRES in two precisions. On hardware where three or more arithmetics are available, which is becoming very common, the number of possible combinations of precisions in GMRES-IR5 is extremely large. We provide an analysis of our theoretical results that identifies a relatively small subset of relevant combinations. By choosing from within this subset one can achieve different levels of tradeoff between cost and robustness, which allows for a finer choice of precisions depending on the problem difficulty and the available hardware. We carry out numerical experiments on random dense and SuiteSparse matrices to validate our theoretical analysis and discuss the complexity of GMRES-IR5.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139756455","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 504-528, March 2024. Abstract. We present a theory for general partial derivatives of matrix functions of the form [math], where [math] is a matrix path of several variables ([math]). Building on results by Mathias [SIAM J. Matrix Anal. Appl., 17 (1996), pp. 610–620] for the first order derivative, we develop a block upper triangular form for higher order partial derivatives. This block form is used to derive conditions for existence and a generalized Daleckiĭ–Kreĭn formula for higher order derivatives. We show that certain specializations of this formula lead to classical formulas of quantum perturbation theory. We show how our results are related to earlier results for higher order Fréchet derivatives. Block forms of complex step approximations are introduced, and we show how those are related to evaluation of derivatives through the upper triangular form. These relations are illustrated with numerical examples.
{"title":"A Unifying Framework for Higher Order Derivatives of Matrix Functions","authors":"Emanuel H. Rubensson","doi":"10.1137/23m1580589","DOIUrl":"https://doi.org/10.1137/23m1580589","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 504-528, March 2024. <br/> Abstract. We present a theory for general partial derivatives of matrix functions of the form [math], where [math] is a matrix path of several variables ([math]). Building on results by Mathias [SIAM J. Matrix Anal. Appl., 17 (1996), pp. 610–620] for the first order derivative, we develop a block upper triangular form for higher order partial derivatives. This block form is used to derive conditions for existence and a generalized Daleckiĭ–Kreĭn formula for higher order derivatives. We show that certain specializations of this formula lead to classical formulas of quantum perturbation theory. We show how our results are related to earlier results for higher order Fréchet derivatives. Block forms of complex step approximations are introduced, and we show how those are related to evaluation of derivatives through the upper triangular form. These relations are illustrated with numerical examples.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139756315","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号