SIAM Review, Volume 66, Issue 3, Page 575-601, May 2024. As key subjects in spectral geometry and combinatorial graph theory, respectively, the (continuous) Hodge Laplacian and the combinatorial Laplacian share similarities in revealing the topological dimension and geometric shape of data and in their realization of diffusion and minimization of harmonic measures. It is believed that they also both associate with vector calculus, through the gradient, curl, and divergence, as argued in the popular usage of “Hodge Laplacians on graphs” in the literature. Nevertheless, these Laplacians are intrinsically different in their domains of definitions and applicability to specific data formats, hindering any in-depth comparison of the two approaches. For example, the spectral decomposition of a vector field on a simple point cloud using combinatorial Laplacians defined on some commonly used simplicial complexes does not give rise to the same curl-free and divergence-free components that one would obtain from the spectral decomposition of a vector field using either the continuous Hodge Laplacians defined on differential forms in manifolds or the discretized Hodge Laplacians defined on a point cloud with boundary in the Eulerian representation or on a regular mesh in the Eulerian representation. To facilitate the comparison and bridge the gap between the combinatorial Laplacian and Hodge Laplacian for the discretization of continuous manifolds with boundary, we further introduce boundary-induced graph (BIG) Laplacians using tools from discrete exterior calculus (DEC). BIG Laplacians are defined on discrete domains with appropriate boundary conditions to characterize the topology and shape of data. The similarities and differences among the combinatorial Laplacian, BIG Laplacian, and Hodge Laplacian are then examined. Through an Eulerian representation of 3D domains as level-set functions on regular grids, we show experimentally the conditions for the convergence of BIG Laplacian eigenvalues to those of the Hodge Laplacian for elementary shapes.
{"title":"Combinatorial and Hodge Laplacians: Similarities and Differences","authors":"Emily Ribando-Gros, Rui Wang, Jiahui Chen, Yiying Tong, Guo-Wei Wei","doi":"10.1137/22m1482299","DOIUrl":"https://doi.org/10.1137/22m1482299","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 575-601, May 2024. <br/> As key subjects in spectral geometry and combinatorial graph theory, respectively, the (continuous) Hodge Laplacian and the combinatorial Laplacian share similarities in revealing the topological dimension and geometric shape of data and in their realization of diffusion and minimization of harmonic measures. It is believed that they also both associate with vector calculus, through the gradient, curl, and divergence, as argued in the popular usage of “Hodge Laplacians on graphs” in the literature. Nevertheless, these Laplacians are intrinsically different in their domains of definitions and applicability to specific data formats, hindering any in-depth comparison of the two approaches. For example, the spectral decomposition of a vector field on a simple point cloud using combinatorial Laplacians defined on some commonly used simplicial complexes does not give rise to the same curl-free and divergence-free components that one would obtain from the spectral decomposition of a vector field using either the continuous Hodge Laplacians defined on differential forms in manifolds or the discretized Hodge Laplacians defined on a point cloud with boundary in the Eulerian representation or on a regular mesh in the Eulerian representation. To facilitate the comparison and bridge the gap between the combinatorial Laplacian and Hodge Laplacian for the discretization of continuous manifolds with boundary, we further introduce boundary-induced graph (BIG) Laplacians using tools from discrete exterior calculus (DEC). BIG Laplacians are defined on discrete domains with appropriate boundary conditions to characterize the topology and shape of data. The similarities and differences among the combinatorial Laplacian, BIG Laplacian, and Hodge Laplacian are then examined. Through an Eulerian representation of 3D domains as level-set functions on regular grids, we show experimentally the conditions for the convergence of BIG Laplacian eigenvalues to those of the Hodge Laplacian for elementary shapes.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"367 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141904538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 3, Page 401-401, May 2024. In “Cardinality Minimization, Constraints, and Regularization: A Survey," Andreas M. Tillmann, Daniel Bienstock, Andrea Lodi, and Alexandra Schwartz consider a class of optimization problems that involve the cardinality of variable vectors in constraints or in the objective function. Such problems have many important applications, e.g., medical imaging (like X-ray tomography), face recognition, wireless sensor network design, stock picking, crystallography, astronomy, computer vision, classification and regression, interpretable machine learning, and statistical data analysis. The emphasis in this paper is on continuous variables, which distinguishes it from a myriad of classical operation research or combinatorial optimization problems. Three general problem classes are studied in detail: cardinality minimization problems, cardinality-constrained problems, and regularized cardinality problems. The paper provides a road map connecting several disciplines and offers an overview of many different computational approaches that are available for cardinality optimization problems. Since such problems are of cross-disciplinary nature, the authors organized their review according to specific application areas and point out overlaps and differences. The paper starts with prominent cardinality optimization problems, namely, signal and image processing, portfolio optimization and management, high-dimensional statistics and machine learning, and some related problems from combinatorics, matrix sparsification, and group/block sparsity. It then continues with exact models and solution methods. The further sections are devoted to relaxations and heuristics, scalability of exact and heuristic algorithms. The authors made a strong effort regarding the organization of their quite long paper, meaning that tables and figures guide the reader to an application or result of interest. In addition, they provide an extensive overview on the literature with more than 400 references.
SIAM Review》，第 66 卷，第 3 期，第 401-401 页，2024 年 5 月。 在 "Cardinality Minimization, Constraints, and Regularization：中，Andreas M. Tillmann、Daniel Bienstock、Andrea Lodi 和 Alexandra Schwartz 考虑了一类优化问题，这些问题涉及约束条件或目标函数中变量向量的万有性。这类问题有很多重要应用，例如医学成像（如 X 射线断层扫描）、人脸识别、无线传感器网络设计、选股、晶体学、天文学、计算机视觉、分类和回归、可解释机器学习以及统计数据分析。本文的重点是连续变量，这使它有别于无数经典的运筹学或组合优化问题。本文详细研究了三类一般问题：卡方最小化问题、卡方受限问题和正则化卡方问题。论文提供了一个连接多个学科的路线图，并概述了可用于万有引力优化问题的多种不同计算方法。由于此类问题具有跨学科性质，作者根据具体应用领域组织了综述，并指出了重叠和差异。论文首先介绍了突出的卡方优化问题，即信号和图像处理、投资组合优化和管理、高维统计和机器学习，以及组合学、矩阵稀疏化和组/块稀疏性中的一些相关问题。然后继续介绍精确模型和求解方法。接下来的章节专门讨论了松弛和启发式算法，以及精确算法和启发式算法的可扩展性。作者在组织篇幅较长的论文方面做出了很大努力，这意味着表格和图表可以引导读者找到感兴趣的应用或结果。此外，他们还提供了 400 多篇参考文献，对文献进行了广泛的概述。
{"title":"Survey and Review","authors":"Marlis Hochbruck","doi":"10.1137/24n97592x","DOIUrl":"https://doi.org/10.1137/24n97592x","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 401-401, May 2024. <br/> In “Cardinality Minimization, Constraints, and Regularization: A Survey,\" Andreas M. Tillmann, Daniel Bienstock, Andrea Lodi, and Alexandra Schwartz consider a class of optimization problems that involve the cardinality of variable vectors in constraints or in the objective function. Such problems have many important applications, e.g., medical imaging (like X-ray tomography), face recognition, wireless sensor network design, stock picking, crystallography, astronomy, computer vision, classification and regression, interpretable machine learning, and statistical data analysis. The emphasis in this paper is on continuous variables, which distinguishes it from a myriad of classical operation research or combinatorial optimization problems. Three general problem classes are studied in detail: cardinality minimization problems, cardinality-constrained problems, and regularized cardinality problems. The paper provides a road map connecting several disciplines and offers an overview of many different computational approaches that are available for cardinality optimization problems. Since such problems are of cross-disciplinary nature, the authors organized their review according to specific application areas and point out overlaps and differences. The paper starts with prominent cardinality optimization problems, namely, signal and image processing, portfolio optimization and management, high-dimensional statistics and machine learning, and some related problems from combinatorics, matrix sparsification, and group/block sparsity. It then continues with exact models and solution methods. The further sections are devoted to relaxations and heuristics, scalability of exact and heuristic algorithms. The authors made a strong effort regarding the organization of their quite long paper, meaning that tables and figures guide the reader to an application or result of interest. In addition, they provide an extensive overview on the literature with more than 400 references.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"367 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141904644","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Andreas M. Tillmann, Daniel Bienstock, Andrea Lodi, Alexandra Schwartz
SIAM Review, Volume 66, Issue 3, Page 403-477, May 2024. We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and we give concrete examples from diverse application fields such as signal and image processing, portfolio selection, and machine learning. The paper discusses general-purpose modeling techniques and broadly applicable as well as problem-specific exact and heuristic solution approaches. While our perspective is that of mathematical optimization, a main goal of this work is to reach out to and build bridges between the different communities in which cardinality optimization problems are frequently encountered. In particular, we highlight that modern mixed-integer programming, which is often regarded as impractical due to the commonly unsatisfactory behavior of black-box solvers applied to generic problem formulations, can in fact produce provably high-quality or even optimal solutions for cardinality optimization problems, even in large-scale real-world settings. Achieving such performance typically involves drawing on the merits of problem-specific knowledge that may stem from different fields of application and, e.g., can shed light on structural properties of a model or its solutions, or can lead to the development of efficient heuristics. We also provide some illustrative examples.
{"title":"Cardinality Minimization, Constraints, and Regularization: A Survey","authors":"Andreas M. Tillmann, Daniel Bienstock, Andrea Lodi, Alexandra Schwartz","doi":"10.1137/21m142770x","DOIUrl":"https://doi.org/10.1137/21m142770x","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 403-477, May 2024. <br/> We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and we give concrete examples from diverse application fields such as signal and image processing, portfolio selection, and machine learning. The paper discusses general-purpose modeling techniques and broadly applicable as well as problem-specific exact and heuristic solution approaches. While our perspective is that of mathematical optimization, a main goal of this work is to reach out to and build bridges between the different communities in which cardinality optimization problems are frequently encountered. In particular, we highlight that modern mixed-integer programming, which is often regarded as impractical due to the commonly unsatisfactory behavior of black-box solvers applied to generic problem formulations, can in fact produce provably high-quality or even optimal solutions for cardinality optimization problems, even in large-scale real-world settings. Achieving such performance typically involves drawing on the merits of problem-specific knowledge that may stem from different fields of application and, e.g., can shed light on structural properties of a model or its solutions, or can lead to the development of efficient heuristics. We also provide some illustrative examples.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"30 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141904540","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 3, Page 533-533, May 2024. The SIGEST article in this issue is “Operator Learning Using Random Features: A Tool for Scientific Computing,” by Nicholas H. Nelsen and Andrew M. Stuart. This work considers the problem of operator learning in infinite-dimensional Banach spaces through the use of random features. The driving application is the approximation of solution operators to partial differential equations (PDEs), here foremost time-dependent problems, that are naturally posed in an infinite-dimensional function space. Typically here, in contrast to the mainstream big data regimes of machine learning applications such as computer vision, high resolution data coming from physical experiments or from computationally expensive simulations of such differential equations is usually small. Fast and approximate surrogates built from such data can be advantageous in building forward models for inverse problems or for doing uncertainty quantification, for instance. Showing how this can be done in infinite dimensions gives rise to approximators which are at the outset resolution and discretization invariant, allowing training on one resolution and deploying on another. At the heart of this work is the function-valued random features methodology that the authors extended from the finite setting of the classical random features approach. Here, the nonlinear operator is approximated by a linear combination of random operators which turn out to be a low-rank approximation and whose computation amounts to a convex, quadratic optimisation problem that is efficiently solvable and for which convergence guarantees can be derived. The methodology is then concretely applied to two concrete PDE examples: Burgers' equations and Darcy flow, demonstrating the applicability of the function-valued random features method, its scalability, discretization invariance, and transferability. The original 2021 article, which appeared in SIAM's Journal on Scientific Computing, has attracted considerable attention. In preparing this SIGEST version, the authors have made numerous modifications and revisions. These include expanding the introductory section and the concluding remarks, condensing the technical content and making it more accessible, and adding a link to an open access GitHub repository that contains all data and code used to produce the results in the paper.
SIAM Review》，第 66 卷第 3 期，第 533-533 页，2024 年 5 月。 本期的 SIGEST 文章是 "Operator Learning Using Random Features：一种科学计算工具"，作者 Nicholas H. Nelsen 和 Andrew M. Stuart。这项研究考虑了通过使用随机特征在无限维巴拿赫空间中进行算子学习的问题。其主要应用是近似偏微分方程（PDEs）的解算子，在这里最重要的是时间相关问题，这些问题自然是在无穷维函数空间中提出的。通常情况下，与计算机视觉等机器学习应用的主流大数据环境不同，来自物理实验或计算成本高昂的此类微分方程模拟的高分辨率数据通常较少。从这些数据中建立快速近似的代用数据，在为逆问题建立前向模型或进行不确定性量化等方面具有优势。通过展示如何在无限维度上实现这一点，我们可以得到从一开始就与分辨率和离散度无关的近似值，从而可以在一种分辨率上进行训练，并在另一种分辨率上进行部署。这项工作的核心是作者从经典随机特征方法的有限设置中扩展出来的函数值随机特征方法。在这里，非线性算子由随机算子的线性组合近似，而随机算子的线性组合是一种低阶近似，其计算相当于一个凸二次优化问题，可高效求解，并可得出收敛保证。然后，我们将这一方法具体应用于两个具体的 PDE 例子：布尔格斯方程和达西流，展示了函数值随机特征方法的适用性、可扩展性、离散不变性和可转移性。最初的 2021 年文章发表在 SIAM 的《科学计算期刊》上，引起了广泛关注。在编写此 SIGEST 版本时，作者进行了大量修改和修订。这些修改和修订包括扩充引言部分和结束语，浓缩技术内容并使其更易于理解，以及添加指向开放访问 GitHub 存储库的链接，该存储库包含用于生成论文结果的所有数据和代码。
{"title":"SIGEST","authors":"The Editors","doi":"10.1137/24n975943","DOIUrl":"https://doi.org/10.1137/24n975943","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 533-533, May 2024. <br/> The SIGEST article in this issue is “Operator Learning Using Random Features: A Tool for Scientific Computing,” by Nicholas H. Nelsen and Andrew M. Stuart. This work considers the problem of operator learning in infinite-dimensional Banach spaces through the use of random features. The driving application is the approximation of solution operators to partial differential equations (PDEs), here foremost time-dependent problems, that are naturally posed in an infinite-dimensional function space. Typically here, in contrast to the mainstream big data regimes of machine learning applications such as computer vision, high resolution data coming from physical experiments or from computationally expensive simulations of such differential equations is usually small. Fast and approximate surrogates built from such data can be advantageous in building forward models for inverse problems or for doing uncertainty quantification, for instance. Showing how this can be done in infinite dimensions gives rise to approximators which are at the outset resolution and discretization invariant, allowing training on one resolution and deploying on another. At the heart of this work is the function-valued random features methodology that the authors extended from the finite setting of the classical random features approach. Here, the nonlinear operator is approximated by a linear combination of random operators which turn out to be a low-rank approximation and whose computation amounts to a convex, quadratic optimisation problem that is efficiently solvable and for which convergence guarantees can be derived. The methodology is then concretely applied to two concrete PDE examples: Burgers' equations and Darcy flow, demonstrating the applicability of the function-valued random features method, its scalability, discretization invariance, and transferability. The original 2021 article, which appeared in SIAM's Journal on Scientific Computing, has attracted considerable attention. In preparing this SIGEST version, the authors have made numerous modifications and revisions. These include expanding the introductory section and the concluding remarks, condensing the technical content and making it more accessible, and adding a link to an open access GitHub repository that contains all data and code used to produce the results in the paper.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"64 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141904646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 2, Page 203-203, May 2024. Inverse problems arise in various applications---for instance, in geoscience, biomedical science, or mining engineering, to mention just a few. The purpose is to recover an object or phenomenon from measured data which is typically subject to noise. The article “Computational Methods for Large-Scale Inverse Problems: A Survey on Hybrid Projection Methods,” by Julianne Chung and Silvia Gazzola, focuses on large, mainly linear, inverse problems. The mathematical modeling of such problems results in a linear system with a very large matrix $A in mathbb{R}^{mtimes n}$ and a perturbed right-hand side. In some applications, it is not even possible to store the matrix, and thus algorithms which only use $A$ in the form of matrix-vector products $Ax$ or $A^Tx$ are the only choice. The article starts with two examples from image deblurring and tomographic reconstruction illustrating the challenges of inverse problems. It then presents the basic idea of regularization which consists of augmenting the model by additional information. Two variants of regularization methods are considered in detail, namely, variational and iterative methods. For variational methods it is crucial to know a good regularization parameter in advance. Unfortunately, its estimation can be expensive. On the other hand, iterative schemes, such as Krylov subspace methods, regularize by early termination of the iterations. Hybrid methods combine these two approaches leveraging the best features of each class. The paper focuses on hybrid projection methods. Here, one starts with a Krylov process in which the original problem is projected onto a low-dimensional subspace. The projected problem is then solved using a variational regularization method. The paper reviews the most relevant direct and iterative regularization techniques before it provides details on the two main building blocks of hybrid methods, namely, generating a subspace for the solution and solving the projected problem. It covers theoretical as well as numerical aspects of these schemes and also presents some extensions of hybrid methods: more general Tikhonov problems, nonstandard projection methods (enrichment, augmentation, recycling), $ell_p$ regularization, Bayesian setting, and nonlinear problems. In addition, relevant software packages are provided. The presentation is very clear and the paper is also readable for those who are not experts in the field. Hence, it is valuable for everyone interested in large-scale inverse problems.
{"title":"Survey and Review","authors":"Marlis Hochbruck","doi":"10.1137/24n975876","DOIUrl":"https://doi.org/10.1137/24n975876","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 203-203, May 2024. <br/> Inverse problems arise in various applications---for instance, in geoscience, biomedical science, or mining engineering, to mention just a few. The purpose is to recover an object or phenomenon from measured data which is typically subject to noise. The article “Computational Methods for Large-Scale Inverse Problems: A Survey on Hybrid Projection Methods,” by Julianne Chung and Silvia Gazzola, focuses on large, mainly linear, inverse problems. The mathematical modeling of such problems results in a linear system with a very large matrix $A in mathbb{R}^{mtimes n}$ and a perturbed right-hand side. In some applications, it is not even possible to store the matrix, and thus algorithms which only use $A$ in the form of matrix-vector products $Ax$ or $A^Tx$ are the only choice. The article starts with two examples from image deblurring and tomographic reconstruction illustrating the challenges of inverse problems. It then presents the basic idea of regularization which consists of augmenting the model by additional information. Two variants of regularization methods are considered in detail, namely, variational and iterative methods. For variational methods it is crucial to know a good regularization parameter in advance. Unfortunately, its estimation can be expensive. On the other hand, iterative schemes, such as Krylov subspace methods, regularize by early termination of the iterations. Hybrid methods combine these two approaches leveraging the best features of each class. The paper focuses on hybrid projection methods. Here, one starts with a Krylov process in which the original problem is projected onto a low-dimensional subspace. The projected problem is then solved using a variational regularization method. The paper reviews the most relevant direct and iterative regularization techniques before it provides details on the two main building blocks of hybrid methods, namely, generating a subspace for the solution and solving the projected problem. It covers theoretical as well as numerical aspects of these schemes and also presents some extensions of hybrid methods: more general Tikhonov problems, nonstandard projection methods (enrichment, augmentation, recycling), $ell_p$ regularization, Bayesian setting, and nonlinear problems. In addition, relevant software packages are provided. The presentation is very clear and the paper is also readable for those who are not experts in the field. Hence, it is valuable for everyone interested in large-scale inverse problems.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"17 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140902936","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 2, Page 368-387, May 2024. This tutorial describes several basic and much-studied types of interactions with incomplete information, analyzing them by means of evolutionary game dynamics. The games include sender-receiver games, owner-challenger contests, costly advertising, and calls for help. We model the evolution of populations of players reacting to each other and compare adaptive dynamics, replicator dynamics, and best-reply dynamics. In particular, we study signaling norms and nonequilibrium outcomes.
{"title":"Dynamics of Signaling Games","authors":"Hannelore De Silva, Karl Sigmund","doi":"10.1137/23m156402x","DOIUrl":"https://doi.org/10.1137/23m156402x","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 368-387, May 2024. <br/> This tutorial describes several basic and much-studied types of interactions with incomplete information, analyzing them by means of evolutionary game dynamics. The games include sender-receiver games, owner-challenger contests, costly advertising, and calls for help. We model the evolution of populations of players reacting to each other and compare adaptive dynamics, replicator dynamics, and best-reply dynamics. In particular, we study signaling norms and nonequilibrium outcomes.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"32 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140903002","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 2, Page 205-284, May 2024. This paper surveys an important class of methods that combine iterative projection methods and variational regularization methods for large-scale inverse problems. Iterative methods such as Krylov subspace methods are invaluable in the numerical linear algebra community and have proved important in solving inverse problems due to their inherent regularizing properties and their ability to handle large-scale problems. Variational regularization describes a broad and important class of methods that are used to obtain reliable solutions to inverse problems, whereby one solves a modified problem that incorporates prior knowledge. Hybrid projection methods combine iterative projection methods with variational regularization techniques in a synergistic way, providing researchers with a powerful computational framework for solving very large inverse problems. Although the idea of a hybrid Krylov method for linear inverse problems goes back to the 1980s, several recent advances on new regularization frameworks and methodologies have made this field ripe for extensions, further analyses, and new applications. In this paper, we provide a practical and accessible introduction to hybrid projection methods in the context of solving large (linear) inverse problems.
{"title":"Computational Methods for Large-Scale Inverse Problems: A Survey on Hybrid Projection Methods","authors":"Julianne Chung, Silvia Gazzola","doi":"10.1137/21m1441420","DOIUrl":"https://doi.org/10.1137/21m1441420","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 205-284, May 2024. <br/> This paper surveys an important class of methods that combine iterative projection methods and variational regularization methods for large-scale inverse problems. Iterative methods such as Krylov subspace methods are invaluable in the numerical linear algebra community and have proved important in solving inverse problems due to their inherent regularizing properties and their ability to handle large-scale problems. Variational regularization describes a broad and important class of methods that are used to obtain reliable solutions to inverse problems, whereby one solves a modified problem that incorporates prior knowledge. Hybrid projection methods combine iterative projection methods with variational regularization techniques in a synergistic way, providing researchers with a powerful computational framework for solving very large inverse problems. Although the idea of a hybrid Krylov method for linear inverse problems goes back to the 1980s, several recent advances on new regularization frameworks and methodologies have made this field ripe for extensions, further analyses, and new applications. In this paper, we provide a practical and accessible introduction to hybrid projection methods in the context of solving large (linear) inverse problems.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"24 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140903028","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, Tong Zhang
SIAM Review, Volume 66, Issue 2, Page 319-352, May 2024. We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving this class of problems converge slowly in practice, involve subproblems that can be as difficult as the original problem, or lack rigorous convergence guarantees. In this paper, we propose a manifold proximal gradient method (ManPG) for solving this class of problems. We prove that the proposed method converges globally to a stationary point and establish its iteration complexity for obtaining an $epsilon$-stationary point. Furthermore, we present numerical results on the sparse PCA and compressed modes problems to demonstrate the advantages of the proposed method. We also discuss some recent advances related to ManPG for Riemannian optimization with nonsmooth objective functions.
{"title":"Nonsmooth Optimization over the Stiefel Manifold and Beyond: Proximal Gradient Method and Recent Variants","authors":"Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, Tong Zhang","doi":"10.1137/24m1628578","DOIUrl":"https://doi.org/10.1137/24m1628578","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 319-352, May 2024. <br/> We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving this class of problems converge slowly in practice, involve subproblems that can be as difficult as the original problem, or lack rigorous convergence guarantees. In this paper, we propose a manifold proximal gradient method (ManPG) for solving this class of problems. We prove that the proposed method converges globally to a stationary point and establish its iteration complexity for obtaining an $epsilon$-stationary point. Furthermore, we present numerical results on the sparse PCA and compressed modes problems to demonstrate the advantages of the proposed method. We also discuss some recent advances related to ManPG for Riemannian optimization with nonsmooth objective functions.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"105 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140902974","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 2, Page 317-317, May 2024. The SIGEST article in this issue is “Nonsmooth Optimization over the Stiefel Manifold and Beyond: Proximal Gradient Method and Recent Variants,” by Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, and Tong Zhang. This work considers nonsmooth optimization on the Stiefel manifold, the manifold of orthonormal $k$-frames in $mathbb{R}^n$. The authors propose a novel proximal gradient algorithm, coined ManPG, for minimizing the sum of a smooth, potentially nonconvex function, and a convex and potentially nonsmooth function whose arguments live on the Stiefel manifold. In contrast to existing approaches, which either are computationally expensive (due to expensive subproblems or slow convergence) or lack rigorous convergence guarantees, ManPG is thoroughly analyzed and features subproblems that can be computed efficiently. Nonsmooth optimization problems on the Stiefel manifold appear in many applications. In statistics sparse principal component analysis (PCA), that is, PCA that seeks principal components with very few nonzero entries, is a prime example. Unsupervised feature selection (machine learning) and blind deconvolution with a sparsity constraint on the deconvolved signal (inverse problems) are important instances of this general objective structure. At the heart of this work is a beautiful interplay between a theoretically well-founded and efficient novel optimization approach for an important class of problems and a set of computational experiments that demonstrate the effectiveness of this new approach. In order to make proximal gradient work for the Stiefel manifold they add a retraction step to the iterations that keeps the iterates feasible. The authors prove global convergence of ManPG to a stationary point and analyze its computational complexity for approximating the latter to $epsilon$ accuracy. The numerical discussion features results for sparse PCA and the problem of computing compressed modes, that is, spatially localized solutions, of the independent-particle Schrödinger equation. The original 2020 article, which appeared in SIAM Journal on Optimization, has attracted considerable attention. In preparing this SIGEST version, the authors have added a discussion on several subsequent works on algorithms for solving Riemannian optimization with nonsmooth objectives. These works were mostly motivated by the ManPG algorithm and include a manifold proximal point algorithm, manifold proximal linear algorithm, stochastic ManPG, zeroth-order ManPG, Riemannian proximal gradient method, and Riemannian proximal Newton method.
{"title":"SIGEST","authors":"The Editors","doi":"10.1137/24n97589x","DOIUrl":"https://doi.org/10.1137/24n97589x","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 317-317, May 2024. <br/> The SIGEST article in this issue is “Nonsmooth Optimization over the Stiefel Manifold and Beyond: Proximal Gradient Method and Recent Variants,” by Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, and Tong Zhang. This work considers nonsmooth optimization on the Stiefel manifold, the manifold of orthonormal $k$-frames in $mathbb{R}^n$. The authors propose a novel proximal gradient algorithm, coined ManPG, for minimizing the sum of a smooth, potentially nonconvex function, and a convex and potentially nonsmooth function whose arguments live on the Stiefel manifold. In contrast to existing approaches, which either are computationally expensive (due to expensive subproblems or slow convergence) or lack rigorous convergence guarantees, ManPG is thoroughly analyzed and features subproblems that can be computed efficiently. Nonsmooth optimization problems on the Stiefel manifold appear in many applications. In statistics sparse principal component analysis (PCA), that is, PCA that seeks principal components with very few nonzero entries, is a prime example. Unsupervised feature selection (machine learning) and blind deconvolution with a sparsity constraint on the deconvolved signal (inverse problems) are important instances of this general objective structure. At the heart of this work is a beautiful interplay between a theoretically well-founded and efficient novel optimization approach for an important class of problems and a set of computational experiments that demonstrate the effectiveness of this new approach. In order to make proximal gradient work for the Stiefel manifold they add a retraction step to the iterations that keeps the iterates feasible. The authors prove global convergence of ManPG to a stationary point and analyze its computational complexity for approximating the latter to $epsilon$ accuracy. The numerical discussion features results for sparse PCA and the problem of computing compressed modes, that is, spatially localized solutions, of the independent-particle Schrödinger equation. The original 2020 article, which appeared in SIAM Journal on Optimization, has attracted considerable attention. In preparing this SIGEST version, the authors have added a discussion on several subsequent works on algorithms for solving Riemannian optimization with nonsmooth objectives. These works were mostly motivated by the ManPG algorithm and include a manifold proximal point algorithm, manifold proximal linear algorithm, stochastic ManPG, zeroth-order ManPG, Riemannian proximal gradient method, and Riemannian proximal Newton method.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"20 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140903048","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Leslie F. Greengard, Shidong Jiang, Manas Rachh, Jun Wang
SIAM Review, Volume 66, Issue 2, Page 287-315, May 2024. We present a new version of the fast Gauss transform (FGT) for discrete and continuous sources. Classical Hermite expansions are avoided entirely, making use only of the plane-wave representation of the Gaussian kernel and a new hierarchical merging scheme. For continuous source distributions sampled on adaptive tensor product grids, we exploit the separable structure of the Gaussian kernel to accelerate the computation. For discrete sources, the scheme relies on the nonuniform fast Fourier transform (NUFFT) to construct near field plane-wave representations. The scheme has been implemented for either free-space or periodic boundary conditions. In many regimes, the speed is comparable to or better than that of the conventional FFT in work per grid point, despite being fully adaptive.
{"title":"A New Version of the Adaptive Fast Gauss Transform for Discrete and Continuous Sources","authors":"Leslie F. Greengard, Shidong Jiang, Manas Rachh, Jun Wang","doi":"10.1137/23m1572453","DOIUrl":"https://doi.org/10.1137/23m1572453","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 287-315, May 2024. <br/> We present a new version of the fast Gauss transform (FGT) for discrete and continuous sources. Classical Hermite expansions are avoided entirely, making use only of the plane-wave representation of the Gaussian kernel and a new hierarchical merging scheme. For continuous source distributions sampled on adaptive tensor product grids, we exploit the separable structure of the Gaussian kernel to accelerate the computation. For discrete sources, the scheme relies on the nonuniform fast Fourier transform (NUFFT) to construct near field plane-wave representations. The scheme has been implemented for either free-space or periodic boundary conditions. In many regimes, the speed is comparable to or better than that of the conventional FFT in work per grid point, despite being fully adaptive.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"27 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140902918","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号