SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 1-23, March 2024. Abstract. The implicit trace estimation problem asks for an approximation of the trace of a square matrix, accessed via matrix-vector products (matvecs). This paper designs new randomized algorithms, XTrace and XNysTrace, for the trace estimation problem by exploiting both variance reduction and the exchangeability principle. For a fixed budget of matvecs, numerical experiments show that the new methods can achieve errors that are orders of magnitude smaller than existing algorithms, such as the Girard–Hutchinson estimator or the Hutch++ estimator. A theoretical analysis confirms the benefits by offering a precise description of the performance of these algorithms as a function of the spectrum of the input matrix. The paper also develops an exchangeable estimator, XDiag, for approximating the diagonal of a square matrix using matvecs.
{"title":"XTrace: Making the Most of Every Sample in Stochastic Trace Estimation","authors":"Ethan N. Epperly, Joel A. Tropp, Robert J. Webber","doi":"10.1137/23m1548323","DOIUrl":"https://doi.org/10.1137/23m1548323","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 1-23, March 2024. <br/> Abstract. The implicit trace estimation problem asks for an approximation of the trace of a square matrix, accessed via matrix-vector products (matvecs). This paper designs new randomized algorithms, XTrace and XNysTrace, for the trace estimation problem by exploiting both variance reduction and the exchangeability principle. For a fixed budget of matvecs, numerical experiments show that the new methods can achieve errors that are orders of magnitude smaller than existing algorithms, such as the Girard–Hutchinson estimator or the Hutch++ estimator. A theoretical analysis confirms the benefits by offering a precise description of the performance of these algorithms as a function of the spectrum of the input matrix. The paper also develops an exchangeable estimator, XDiag, for approximating the diagonal of a square matrix using matvecs.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139094693","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ming Zhou, Merico Argentati, Andrew V. Knyazev, Klaus Neymeyr
SIAM Journal on Matrix Analysis and Applications, Volume 44, Issue 4, Page 1852-1878, December 2023. Abstract. Convergence analysis of block iterative solvers for Hermitian eigenvalue problems and closely related research on properties of matrix-based signal filters are challenging and are attracting increased attention due to their recent applications in spectral data clustering and graph-based signal processing. We combine majorization-based techniques pioneered for investigating the Rayleigh–Ritz method in [A. V. Knyazev and M. E. Argentati, SIAM J. Matrix Anal. Appl., 31 (2010), pp. 1521–1537] with tools of classical analysis of the block power method by Rutishauser [Numer. Math., 13 (1969), pp. 4–13] to derive sharp convergence rate bounds of abstract block iterations, wherein tuples of tangents of principal angles or relative errors of Ritz values are bounded using majorization in terms of arranged partial sums and tuples of convergence factors. Our novel bounds are robust in the presence of clusters of eigenvalues, improve previous results, and are applicable to most known block iterative solvers and matrix-based filters, e.g., to block power, Chebyshev, and Lanczos methods combined with polynomial filtering. The sharpness of our bounds is fundamental, implying that the bounds cannot be improved without further assumptions.
{"title":"Sharp Majorization-Type Cluster Robust Bounds for Block Filters and Eigensolvers","authors":"Ming Zhou, Merico Argentati, Andrew V. Knyazev, Klaus Neymeyr","doi":"10.1137/23m1551729","DOIUrl":"https://doi.org/10.1137/23m1551729","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 44, Issue 4, Page 1852-1878, December 2023. <br/> Abstract. Convergence analysis of block iterative solvers for Hermitian eigenvalue problems and closely related research on properties of matrix-based signal filters are challenging and are attracting increased attention due to their recent applications in spectral data clustering and graph-based signal processing. We combine majorization-based techniques pioneered for investigating the Rayleigh–Ritz method in [A. V. Knyazev and M. E. Argentati, SIAM J. Matrix Anal. Appl., 31 (2010), pp. 1521–1537] with tools of classical analysis of the block power method by Rutishauser [Numer. Math., 13 (1969), pp. 4–13] to derive sharp convergence rate bounds of abstract block iterations, wherein tuples of tangents of principal angles or relative errors of Ritz values are bounded using majorization in terms of arranged partial sums and tuples of convergence factors. Our novel bounds are robust in the presence of clusters of eigenvalues, improve previous results, and are applicable to most known block iterative solvers and matrix-based filters, e.g., to block power, Chebyshev, and Lanczos methods combined with polynomial filtering. The sharpness of our bounds is fundamental, implying that the bounds cannot be improved without further assumptions.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2023-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138513869","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Matrix Analysis and Applications, Volume 44, Issue 4, Page 1822-1851, December 2023. Abstract. We provide a characterization of the continuous-time Markov models where the Markov matrices from the model can be parameterized directly in terms of the associated rate matrices (generators). That is, each Markov matrix can be expressed as the sum of the identity matrix and a rate matrix from the model. We show that the existence of an underlying Jordan algebra provides a sufficient condition, which becomes necessary for (so-called) linear models. We connect this property to the well-known uniformization procedure for continuous-time Markov chains by demonstrating that the property is equivalent to all Markov matrices from the model taking the same form as the corresponding discrete-time Markov matrices in the uniformized process. We apply our results to analyze two model hierarchies practically important to phylogenetic inference, obtained by assuming (i) time reversibility and (ii) permutation symmetry, respectively.
{"title":"Uniformization Stable Markov Models and Their Jordan Algebraic Structure","authors":"Luke Cooper, Jeremy Sumner","doi":"10.1137/22m1474527","DOIUrl":"https://doi.org/10.1137/22m1474527","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 44, Issue 4, Page 1822-1851, December 2023. <br/> Abstract. We provide a characterization of the continuous-time Markov models where the Markov matrices from the model can be parameterized directly in terms of the associated rate matrices (generators). That is, each Markov matrix can be expressed as the sum of the identity matrix and a rate matrix from the model. We show that the existence of an underlying Jordan algebra provides a sufficient condition, which becomes necessary for (so-called) linear models. We connect this property to the well-known uniformization procedure for continuous-time Markov chains by demonstrating that the property is equivalent to all Markov matrices from the model taking the same form as the corresponding discrete-time Markov matrices in the uniformized process. We apply our results to analyze two model hierarchies practically important to phylogenetic inference, obtained by assuming (i) time reversibility and (ii) permutation symmetry, respectively.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2023-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138513873","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Matrix Analysis and Applications, Volume 44, Issue 4, Page 1879-1907, December 2023. Abstract. We solve the open problem of describing the possible Kronecker invariants of quasi-regular matrix pencils under bounded rank perturbations. By a quasi-regular matrix pencil we mean the full (normal) rank matrix pencil. The solution is explicit and constructive, and it is valid over arbitrary fields.
{"title":"Bounded Rank Perturbations of Quasi-Regular Pencils Over Arbitrary Fields","authors":"Marija Dodig, Marko Stošić","doi":"10.1137/22m1504068","DOIUrl":"https://doi.org/10.1137/22m1504068","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 44, Issue 4, Page 1879-1907, December 2023. <br/> Abstract. We solve the open problem of describing the possible Kronecker invariants of quasi-regular matrix pencils under bounded rank perturbations. By a quasi-regular matrix pencil we mean the full (normal) rank matrix pencil. The solution is explicit and constructive, and it is valid over arbitrary fields.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2023-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138513875","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Philipp Dettling, Roser Homs, Carlos Améndola, Mathias Drton, Niels Richard Hansen
SIAM Journal on Matrix Analysis and Applications, Volume 44, Issue 4, Page 1799-1821, December 2023. Abstract. The recently introduced graphical continuous Lyapunov models provide a new approach to statistical modeling of correlated multivariate data. The models view each observation as a one-time cross-sectional snapshot of a multivariate dynamic process in equilibrium. The covariance matrix for the data is obtained by solving a continuous Lyapunov equation that is parametrized by the drift matrix of the dynamic process. In this context, different statistical models postulate different sparsity patterns in the drift matrix, and it becomes a crucial problem to clarify whether a given sparsity assumption allows one to uniquely recover the drift matrix parameters from the covariance matrix of the data. We study this identifiability problem by representing sparsity patterns by directed graphs. Our main result proves that the drift matrix is globally identifiable if and only if the graph for the sparsity pattern is simple (i.e., does not contain directed 2-cycles). Moreover, we present a necessary condition for generic identifiability and provide a computational classification of small graphs with up to 5 nodes.
{"title":"Identifiability in Continuous Lyapunov Models","authors":"Philipp Dettling, Roser Homs, Carlos Améndola, Mathias Drton, Niels Richard Hansen","doi":"10.1137/22m1520311","DOIUrl":"https://doi.org/10.1137/22m1520311","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 44, Issue 4, Page 1799-1821, December 2023. <br/> Abstract. The recently introduced graphical continuous Lyapunov models provide a new approach to statistical modeling of correlated multivariate data. The models view each observation as a one-time cross-sectional snapshot of a multivariate dynamic process in equilibrium. The covariance matrix for the data is obtained by solving a continuous Lyapunov equation that is parametrized by the drift matrix of the dynamic process. In this context, different statistical models postulate different sparsity patterns in the drift matrix, and it becomes a crucial problem to clarify whether a given sparsity assumption allows one to uniquely recover the drift matrix parameters from the covariance matrix of the data. We study this identifiability problem by representing sparsity patterns by directed graphs. Our main result proves that the drift matrix is globally identifiable if and only if the graph for the sparsity pattern is simple (i.e., does not contain directed 2-cycles). Moreover, we present a necessary condition for generic identifiability and provide a computational classification of small graphs with up to 5 nodes.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2023-12-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138513865","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Matrix Analysis and Applications, Volume 44, Issue 4, Page 1771-1798, December 2023. Abstract. Solving the linear system [math] is often the major computational burden when a forward-backward evolutionary equation must be solved in a problem, where [math] is the so-called all-at-once matrix of the forward subproblem after space-time discretization. An efficient solver requires a good preconditioner for [math]. Inspired by the structure of [math], we precondition [math] by [math] with [math] being a block [math]-circulant matrix constructed by replacing the Toeplitz matrices in [math] by the [math]-circulant matrices. By a block Fourier diagonalization of [math], the computation of the preconditioning step [math] is parallelizable for all the time steps. We give a spectral analysis for the preconditioned matrix [math] and prove that for any one-step stable time-integrator the eigenvalues of [math] spread in a mesh-independent interval [math] if the parameter [math] weakly scales in terms of the number of time steps [math] as [math], where [math] is a free constant. Two applications of the proposed preconditioner are illustrated: PDE-constrained optimal control problems and parabolic source identification problems. Numerical results for both problems indicate that spectral analysis predicts the convergence rate of the preconditioned conjugate gradient method very well.
{"title":"PinT Preconditioner for Forward-Backward Evolutionary Equations","authors":"Shu-Lin Wu, Zhiyong Wang, Tao Zhou","doi":"10.1137/22m1516476","DOIUrl":"https://doi.org/10.1137/22m1516476","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 44, Issue 4, Page 1771-1798, December 2023. <br/> Abstract. Solving the linear system [math] is often the major computational burden when a forward-backward evolutionary equation must be solved in a problem, where [math] is the so-called all-at-once matrix of the forward subproblem after space-time discretization. An efficient solver requires a good preconditioner for [math]. Inspired by the structure of [math], we precondition [math] by [math] with [math] being a block [math]-circulant matrix constructed by replacing the Toeplitz matrices in [math] by the [math]-circulant matrices. By a block Fourier diagonalization of [math], the computation of the preconditioning step [math] is parallelizable for all the time steps. We give a spectral analysis for the preconditioned matrix [math] and prove that for any one-step stable time-integrator the eigenvalues of [math] spread in a mesh-independent interval [math] if the parameter [math] weakly scales in terms of the number of time steps [math] as [math], where [math] is a free constant. Two applications of the proposed preconditioner are illustrated: PDE-constrained optimal control problems and parabolic source identification problems. Numerical results for both problems indicate that spectral analysis predicts the convergence rate of the preconditioned conjugate gradient method very well.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2023-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138513874","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We show that the global minimum solution of can be found in closed form with singular value decompositions and generalized singular value decompositions for a variety of constraints on involving rank, norm, symmetry, two-sided product, and prescribed eigenvalue. This extends the solution of Friedland–Torokhti for the generalized rank-constrained approximation problem to other constraints and provides an alternative solution for rank constraint in terms of singular value decompositions. For more complicated constraints on involving structures such as Toeplitz, Hankel, circulant, nonnegativity, stochasticity, positive semidefiniteness, prescribed eigenvector, etc., we prove that a simple iterative method is linearly and globally convergent to the global minimum solution.
{"title":"Generalized Matrix Nearness Problems","authors":"Zihao Li, Lek-Heng Lim","doi":"10.1137/22m1526034","DOIUrl":"https://doi.org/10.1137/22m1526034","url":null,"abstract":"We show that the global minimum solution of can be found in closed form with singular value decompositions and generalized singular value decompositions for a variety of constraints on involving rank, norm, symmetry, two-sided product, and prescribed eigenvalue. This extends the solution of Friedland–Torokhti for the generalized rank-constrained approximation problem to other constraints and provides an alternative solution for rank constraint in terms of singular value decompositions. For more complicated constraints on involving structures such as Toeplitz, Hankel, circulant, nonnegativity, stochasticity, positive semidefiniteness, prescribed eigenvector, etc., we prove that a simple iterative method is linearly and globally convergent to the global minimum solution.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135137335","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Philip A. Knight, Luce le Gorrec, Sandrine Mouysset, Daniel Ruiz
{"title":"Introducing the Class of SemiDoubly Stochastic Matrices: A Novel Scaling Approach for Rectangular Matrices","authors":"Philip A. Knight, Luce le Gorrec, Sandrine Mouysset, Daniel Ruiz","doi":"10.1137/22m1519791","DOIUrl":"https://doi.org/10.1137/22m1519791","url":null,"abstract":"","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135137337","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Stochastic Algebraic Riccati Equations Are Almost as Easy as Deterministic Ones Theoretically","authors":"Zhen-Chen Guo, Xin Liang","doi":"10.1137/22m1514647","DOIUrl":"https://doi.org/10.1137/22m1514647","url":null,"abstract":"","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135137334","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mareike Dressler, André Uschmajew, Venkat Chandrasekaran
The decomposition or approximation of a linear operator on a matrix space as a sum of Kronecker products plays an important role in matrix equations and low-rank modeling. The approximation problem in Frobenius norm admits a well-known solution via the singular value decomposition. However, the approximation problem in spectral norm, which is more natural for linear operators, is much more challenging. In particular, the Frobenius norm solution can be far from optimal in spectral norm. We describe an alternating optimization method based on semidefinite programming to obtain high-quality approximations in spectral norm, and we present computational experiments to illustrate the advantages of our approach.
{"title":"Kronecker Product Approximation of Operators in Spectral Norm via Alternating SDP","authors":"Mareike Dressler, André Uschmajew, Venkat Chandrasekaran","doi":"10.1137/22m1509953","DOIUrl":"https://doi.org/10.1137/22m1509953","url":null,"abstract":"The decomposition or approximation of a linear operator on a matrix space as a sum of Kronecker products plays an important role in matrix equations and low-rank modeling. The approximation problem in Frobenius norm admits a well-known solution via the singular value decomposition. However, the approximation problem in spectral norm, which is more natural for linear operators, is much more challenging. In particular, the Frobenius norm solution can be far from optimal in spectral norm. We describe an alternating optimization method based on semidefinite programming to obtain high-quality approximations in spectral norm, and we present computational experiments to illustrate the advantages of our approach.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135192437","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号