Liliana Borcea, Josselin Garnier, Alexander V. Mamonov, Jörn Zimmerling
SIAM Review, Volume 66, Issue 3, Page 501-532, May 2024. Waveform inversion is concerned with estimating a heterogeneous medium, modeled by variable coefficients of wave equations, using sources that emit probing signals and receivers that record the generated waves. It is an old and intensively studied inverse problem with a wide range of applications, but the existing inversion methodologies are still far from satisfactory. The typical mathematical formulation is a nonlinear least squares data fit optimization and the difficulty stems from the nonconvexity of the objective function that displays numerous local minima at which local optimization approaches stagnate. This pathological behavior has at least three unavoidable causes: (1) The mapping from the unknown coefficients to the wave field is nonlinear and complicated. (2) The sources and receivers typically lie on a single side of the medium, so only backscattered waves are measured. (3) The probing signals are band limited and with high frequency content. There is a lot of activity in the computational science and engineering communities that seeks to mitigate the difficulty of estimating the medium by data fitting. In this paper we present a different point of view, based on reduced order models (ROMs) of two operators that control the wave propagation. The ROMs are called data driven because they are computed directly from the measurements, without any knowledge of the wave field inside the inaccessible medium. This computation is noniterative and uses standard numerical linear algebra methods. The resulting ROMs capture features of the physics of wave propagation in a complementary way and have surprisingly good approximation properties that facilitate waveform inversion.
{"title":"When Data Driven Reduced Order Modeling Meets Full Waveform Inversion","authors":"Liliana Borcea, Josselin Garnier, Alexander V. Mamonov, Jörn Zimmerling","doi":"10.1137/23m1552826","DOIUrl":"https://doi.org/10.1137/23m1552826","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 501-532, May 2024. <br/> Waveform inversion is concerned with estimating a heterogeneous medium, modeled by variable coefficients of wave equations, using sources that emit probing signals and receivers that record the generated waves. It is an old and intensively studied inverse problem with a wide range of applications, but the existing inversion methodologies are still far from satisfactory. The typical mathematical formulation is a nonlinear least squares data fit optimization and the difficulty stems from the nonconvexity of the objective function that displays numerous local minima at which local optimization approaches stagnate. This pathological behavior has at least three unavoidable causes: (1) The mapping from the unknown coefficients to the wave field is nonlinear and complicated. (2) The sources and receivers typically lie on a single side of the medium, so only backscattered waves are measured. (3) The probing signals are band limited and with high frequency content. There is a lot of activity in the computational science and engineering communities that seeks to mitigate the difficulty of estimating the medium by data fitting. In this paper we present a different point of view, based on reduced order models (ROMs) of two operators that control the wave propagation. The ROMs are called data driven because they are computed directly from the measurements, without any knowledge of the wave field inside the inaccessible medium. This computation is noniterative and uses standard numerical linear algebra methods. The resulting ROMs capture features of the physics of wave propagation in a complementary way and have surprisingly good approximation properties that facilitate waveform inversion.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141904641","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 3, Page 479-479, May 2024. Equitable distribution of geographically dispersed resources presents a significant challenge, particularly in defining quantifiable measures of equity. How can we optimally allocate polling sites or hospitals to serve their constituencies? This issue's first Research Spotlight, “Persistent Homology for Resource Coverage: A Case Study of Access to Polling Sites," addresses these questions by demonstrating the application of topological data analysis to identify holes in resource accessibility and coverage. Authors Abigail Hickok, Benjamin Jarman, Michael Johnson, Jiajie Luo, and Mason A. Porter employ persistent homology, a technique that tracks the formation and disappearance of these holes as spatial scales vary. To make matters concrete, the authors consider a case study on access to polling sites and use a non-Euclidean distance that accounts for both travel and waiting times. In their case study, the authors use a weighted Vietoris--Rips filtration based on a symmetrized form of this distance and limit their examination to instances where the approximations underlying the filtration are less likely to lead to approximation-based artifacts. Details, as well as source code, are provided on the estimation of the various quantities, such as travel time, waiting time, and demographics (e.g., age, vehicle access). The result is a homology class that “dies" at time $t$ if it takes $t$ total minutes to cast a vote. The paper concludes with an exposition of potential limitations and future directions that serve to encourage additional investigation into this class of problems (which includes settings where one wants to deploy different sensors to cover a spatial domain) and related techniques. What secrets lurk within? From flaws in human-made infrastructure to materials deep beneath the Earth's land and ocean surfaces to anomalies in patients, our next Research Spotlight, “When Data Driven Reduced Order Modeling Meets Full Waveform Inversion," addresses math and methods to recover the unknown. Authors Liliana Borcea, Josselin Garnier, Alexander V. Mamonov, and Jörn Zimmerling show how tools from numerical linear algebra and reduced-order modeling can be brought to bear on inverse wave scattering problems. Their setup encapsulates a wide variety of sensing modalities, wherein receivers emit a signal (such as an acoustic wave) and a time series of wavefield measurements is subsequently captured at one or more sources. Full waveform inversion refers to the recovery of the unknown “within" and is typically addressed via iterative, nonlinear equations/least-squares solvers. However, it is often plagued by a notoriously nonconvex, ill-conditioned optimization landscape. The authors show how some of the challenges typically encountered in this inversion can be mitigated with the use of reduced-order models. These models employ observed data snapshots to form lower-dimensional, computationally a
{"title":"Research Spotlights","authors":"Stefan M. Wild","doi":"10.1137/24n975931","DOIUrl":"https://doi.org/10.1137/24n975931","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 479-479, May 2024. <br/> Equitable distribution of geographically dispersed resources presents a significant challenge, particularly in defining quantifiable measures of equity. How can we optimally allocate polling sites or hospitals to serve their constituencies? This issue's first Research Spotlight, “Persistent Homology for Resource Coverage: A Case Study of Access to Polling Sites,\" addresses these questions by demonstrating the application of topological data analysis to identify holes in resource accessibility and coverage. Authors Abigail Hickok, Benjamin Jarman, Michael Johnson, Jiajie Luo, and Mason A. Porter employ persistent homology, a technique that tracks the formation and disappearance of these holes as spatial scales vary. To make matters concrete, the authors consider a case study on access to polling sites and use a non-Euclidean distance that accounts for both travel and waiting times. In their case study, the authors use a weighted Vietoris--Rips filtration based on a symmetrized form of this distance and limit their examination to instances where the approximations underlying the filtration are less likely to lead to approximation-based artifacts. Details, as well as source code, are provided on the estimation of the various quantities, such as travel time, waiting time, and demographics (e.g., age, vehicle access). The result is a homology class that “dies\" at time $t$ if it takes $t$ total minutes to cast a vote. The paper concludes with an exposition of potential limitations and future directions that serve to encourage additional investigation into this class of problems (which includes settings where one wants to deploy different sensors to cover a spatial domain) and related techniques. What secrets lurk within? From flaws in human-made infrastructure to materials deep beneath the Earth's land and ocean surfaces to anomalies in patients, our next Research Spotlight, “When Data Driven Reduced Order Modeling Meets Full Waveform Inversion,\" addresses math and methods to recover the unknown. Authors Liliana Borcea, Josselin Garnier, Alexander V. Mamonov, and Jörn Zimmerling show how tools from numerical linear algebra and reduced-order modeling can be brought to bear on inverse wave scattering problems. Their setup encapsulates a wide variety of sensing modalities, wherein receivers emit a signal (such as an acoustic wave) and a time series of wavefield measurements is subsequently captured at one or more sources. Full waveform inversion refers to the recovery of the unknown “within\" and is typically addressed via iterative, nonlinear equations/least-squares solvers. However, it is often plagued by a notoriously nonconvex, ill-conditioned optimization landscape. The authors show how some of the challenges typically encountered in this inversion can be mitigated with the use of reduced-order models. These models employ observed data snapshots to form lower-dimensional, computationally a","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141908890","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Abigail Hickok, Benjamin Jarman, Michael Johnson, Jiajie Luo, Mason A. Porter
SIAM Review, Volume 66, Issue 3, Page 481-500, May 2024. It is important to choose the geographical distributions of public resources in a fair and equitable manner. However, it is complicated to quantify the equity of such a distribution; important factors include distances to resource sites, availability of transportation, and ease of travel. We use persistent homology, which is a tool from topological data analysis, to study the availability and coverage of polling sites. The information from persistent homology allows us to infer holes in a distribution of polling sites. We analyze and compare the coverage of polling sites in Los Angeles County and five cities (Atlanta, Chicago, Jacksonville, New York City, and Salt Lake City), and we conclude that computation of persistent homology appears to be a reasonable approach to analyzing resource coverage.
{"title":"Persistent Homology for Resource Coverage: A Case Study of Access to Polling Sites","authors":"Abigail Hickok, Benjamin Jarman, Michael Johnson, Jiajie Luo, Mason A. Porter","doi":"10.1137/22m150410x","DOIUrl":"https://doi.org/10.1137/22m150410x","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 481-500, May 2024. <br/> It is important to choose the geographical distributions of public resources in a fair and equitable manner. However, it is complicated to quantify the equity of such a distribution; important factors include distances to resource sites, availability of transportation, and ease of travel. We use persistent homology, which is a tool from topological data analysis, to study the availability and coverage of polling sites. The information from persistent homology allows us to infer holes in a distribution of polling sites. We analyze and compare the coverage of polling sites in Los Angeles County and five cities (Atlanta, Chicago, Jacksonville, New York City, and Salt Lake City), and we conclude that computation of persistent homology appears to be a reasonable approach to analyzing resource coverage.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141909213","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 3, Page 573-573, May 2024. In this issue the Education section presents “Combinatorial and Hodge Laplacians: Similarities and Differences,” by Emily Ribando-Gros, Rui Wang, Jiahui Chen, Yiying Tong, and Guo-Wei Wei. Combinatorial Laplacians and their spectra are important tools in the study of molecular stability, electrical networks, neuroscience, deep learning, signal processing, etc. The continuous Hodge Laplacian allows one, in some cases, to generate an unknown shape from only its Laplacian spectrum. In particular, both combinatorial Laplacians and continuous Hodge Laplacian are useful in describing the topology of data; see, for instance, [L.-H. Lim, “Hodge Laplacians on graphs,” SIAM Rev., 62 (2020), pp. 685--715]. Since nowadays computations frequently involve these Laplacians, it is important to have a good understanding of the differences and relations between them. Indeed, though the Hodge Laplacian and the combinatorial Laplacian share similarities in revealing the topological dimension and geometric shape of data, at the same time they are intrinsically different in their domains of definitions and applicability to specific data formats. To facilitate comparisons, the authors introduce boundary-induced graph (BIG) Laplacians, the purpose of which is “to put the combinatorial Laplacians and Hodge Laplacian on equal footing.” BIG Laplacian brings, in fact, the combinatorial Laplacian closer to the continuous Hodge Laplacian. In this paper similarities and differences between combinatorial Laplacian, BIG Laplacian, and Hodge Laplacian are examined. Some elements of spectral analysis related to topological data analysis (TDA) are also provided. TDA and connected ideas have recently gained a lot of interest, and so this paper is timely. It is written in a way that should make it accessible for early career researchers; the reader should already have a good understanding of some notions of graph theory, spectral geometry, differential geometry, and algebraic topology. The paper is not self-contained and eventually could be used by group-based research projects in a Master's program for advanced mathematics students.
{"title":"Education","authors":"Hélène Frankowska","doi":"10.1137/24n975955","DOIUrl":"https://doi.org/10.1137/24n975955","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 573-573, May 2024. <br/> In this issue the Education section presents “Combinatorial and Hodge Laplacians: Similarities and Differences,” by Emily Ribando-Gros, Rui Wang, Jiahui Chen, Yiying Tong, and Guo-Wei Wei. Combinatorial Laplacians and their spectra are important tools in the study of molecular stability, electrical networks, neuroscience, deep learning, signal processing, etc. The continuous Hodge Laplacian allows one, in some cases, to generate an unknown shape from only its Laplacian spectrum. In particular, both combinatorial Laplacians and continuous Hodge Laplacian are useful in describing the topology of data; see, for instance, [L.-H. Lim, “Hodge Laplacians on graphs,” SIAM Rev., 62 (2020), pp. 685--715]. Since nowadays computations frequently involve these Laplacians, it is important to have a good understanding of the differences and relations between them. Indeed, though the Hodge Laplacian and the combinatorial Laplacian share similarities in revealing the topological dimension and geometric shape of data, at the same time they are intrinsically different in their domains of definitions and applicability to specific data formats. To facilitate comparisons, the authors introduce boundary-induced graph (BIG) Laplacians, the purpose of which is “to put the combinatorial Laplacians and Hodge Laplacian on equal footing.” BIG Laplacian brings, in fact, the combinatorial Laplacian closer to the continuous Hodge Laplacian. In this paper similarities and differences between combinatorial Laplacian, BIG Laplacian, and Hodge Laplacian are examined. Some elements of spectral analysis related to topological data analysis (TDA) are also provided. TDA and connected ideas have recently gained a lot of interest, and so this paper is timely. It is written in a way that should make it accessible for early career researchers; the reader should already have a good understanding of some notions of graph theory, spectral geometry, differential geometry, and algebraic topology. The paper is not self-contained and eventually could be used by group-based research projects in a Master's program for advanced mathematics students.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141909212","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 3, Page 535-571, May 2024. Supervised operator learning centers on the use of training data, in the form of input-output pairs, to estimate maps between infinite-dimensional spaces. It is emerging as a powerful tool to complement traditional scientific computing, which may often be framed in terms of operators mapping between spaces of functions. Building on the classical random features methodology for scalar regression, this paper introduces the function-valued random features method. This leads to a supervised operator learning architecture that is practical for nonlinear problems yet is structured enough to facilitate efficient training through the optimization of a convex, quadratic cost. Due to the quadratic structure, the trained model is equipped with convergence guarantees and error and complexity bounds, properties that are not readily available for most other operator learning architectures. At its core, the proposed approach builds a linear combination of random operators. This turns out to be a low-rank approximation of an operator-valued kernel ridge regression algorithm, and hence the method also has strong connections to Gaussian process regression. The paper designs function-valued random features that are tailored to the structure of two nonlinear operator learning benchmark problems arising from parametric partial differential equations. Numerical results demonstrate the scalability, discretization invariance, and transferability of the function-valued random features method.
{"title":"Operator Learning Using Random Features: A Tool for Scientific Computing","authors":"Nicholas H. Nelsen, Andrew M. Stuart","doi":"10.1137/24m1648703","DOIUrl":"https://doi.org/10.1137/24m1648703","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 535-571, May 2024. <br/> Supervised operator learning centers on the use of training data, in the form of input-output pairs, to estimate maps between infinite-dimensional spaces. It is emerging as a powerful tool to complement traditional scientific computing, which may often be framed in terms of operators mapping between spaces of functions. Building on the classical random features methodology for scalar regression, this paper introduces the function-valued random features method. This leads to a supervised operator learning architecture that is practical for nonlinear problems yet is structured enough to facilitate efficient training through the optimization of a convex, quadratic cost. Due to the quadratic structure, the trained model is equipped with convergence guarantees and error and complexity bounds, properties that are not readily available for most other operator learning architectures. At its core, the proposed approach builds a linear combination of random operators. This turns out to be a low-rank approximation of an operator-valued kernel ridge regression algorithm, and hence the method also has strong connections to Gaussian process regression. The paper designs function-valued random features that are tailored to the structure of two nonlinear operator learning benchmark problems arising from parametric partial differential equations. Numerical results demonstrate the scalability, discretization invariance, and transferability of the function-valued random features method.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141909214","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 3, Page 575-601, May 2024. As key subjects in spectral geometry and combinatorial graph theory, respectively, the (continuous) Hodge Laplacian and the combinatorial Laplacian share similarities in revealing the topological dimension and geometric shape of data and in their realization of diffusion and minimization of harmonic measures. It is believed that they also both associate with vector calculus, through the gradient, curl, and divergence, as argued in the popular usage of “Hodge Laplacians on graphs” in the literature. Nevertheless, these Laplacians are intrinsically different in their domains of definitions and applicability to specific data formats, hindering any in-depth comparison of the two approaches. For example, the spectral decomposition of a vector field on a simple point cloud using combinatorial Laplacians defined on some commonly used simplicial complexes does not give rise to the same curl-free and divergence-free components that one would obtain from the spectral decomposition of a vector field using either the continuous Hodge Laplacians defined on differential forms in manifolds or the discretized Hodge Laplacians defined on a point cloud with boundary in the Eulerian representation or on a regular mesh in the Eulerian representation. To facilitate the comparison and bridge the gap between the combinatorial Laplacian and Hodge Laplacian for the discretization of continuous manifolds with boundary, we further introduce boundary-induced graph (BIG) Laplacians using tools from discrete exterior calculus (DEC). BIG Laplacians are defined on discrete domains with appropriate boundary conditions to characterize the topology and shape of data. The similarities and differences among the combinatorial Laplacian, BIG Laplacian, and Hodge Laplacian are then examined. Through an Eulerian representation of 3D domains as level-set functions on regular grids, we show experimentally the conditions for the convergence of BIG Laplacian eigenvalues to those of the Hodge Laplacian for elementary shapes.
{"title":"Combinatorial and Hodge Laplacians: Similarities and Differences","authors":"Emily Ribando-Gros, Rui Wang, Jiahui Chen, Yiying Tong, Guo-Wei Wei","doi":"10.1137/22m1482299","DOIUrl":"https://doi.org/10.1137/22m1482299","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 575-601, May 2024. <br/> As key subjects in spectral geometry and combinatorial graph theory, respectively, the (continuous) Hodge Laplacian and the combinatorial Laplacian share similarities in revealing the topological dimension and geometric shape of data and in their realization of diffusion and minimization of harmonic measures. It is believed that they also both associate with vector calculus, through the gradient, curl, and divergence, as argued in the popular usage of “Hodge Laplacians on graphs” in the literature. Nevertheless, these Laplacians are intrinsically different in their domains of definitions and applicability to specific data formats, hindering any in-depth comparison of the two approaches. For example, the spectral decomposition of a vector field on a simple point cloud using combinatorial Laplacians defined on some commonly used simplicial complexes does not give rise to the same curl-free and divergence-free components that one would obtain from the spectral decomposition of a vector field using either the continuous Hodge Laplacians defined on differential forms in manifolds or the discretized Hodge Laplacians defined on a point cloud with boundary in the Eulerian representation or on a regular mesh in the Eulerian representation. To facilitate the comparison and bridge the gap between the combinatorial Laplacian and Hodge Laplacian for the discretization of continuous manifolds with boundary, we further introduce boundary-induced graph (BIG) Laplacians using tools from discrete exterior calculus (DEC). BIG Laplacians are defined on discrete domains with appropriate boundary conditions to characterize the topology and shape of data. The similarities and differences among the combinatorial Laplacian, BIG Laplacian, and Hodge Laplacian are then examined. Through an Eulerian representation of 3D domains as level-set functions on regular grids, we show experimentally the conditions for the convergence of BIG Laplacian eigenvalues to those of the Hodge Laplacian for elementary shapes.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141904538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 3, Page 401-401, May 2024. In “Cardinality Minimization, Constraints, and Regularization: A Survey," Andreas M. Tillmann, Daniel Bienstock, Andrea Lodi, and Alexandra Schwartz consider a class of optimization problems that involve the cardinality of variable vectors in constraints or in the objective function. Such problems have many important applications, e.g., medical imaging (like X-ray tomography), face recognition, wireless sensor network design, stock picking, crystallography, astronomy, computer vision, classification and regression, interpretable machine learning, and statistical data analysis. The emphasis in this paper is on continuous variables, which distinguishes it from a myriad of classical operation research or combinatorial optimization problems. Three general problem classes are studied in detail: cardinality minimization problems, cardinality-constrained problems, and regularized cardinality problems. The paper provides a road map connecting several disciplines and offers an overview of many different computational approaches that are available for cardinality optimization problems. Since such problems are of cross-disciplinary nature, the authors organized their review according to specific application areas and point out overlaps and differences. The paper starts with prominent cardinality optimization problems, namely, signal and image processing, portfolio optimization and management, high-dimensional statistics and machine learning, and some related problems from combinatorics, matrix sparsification, and group/block sparsity. It then continues with exact models and solution methods. The further sections are devoted to relaxations and heuristics, scalability of exact and heuristic algorithms. The authors made a strong effort regarding the organization of their quite long paper, meaning that tables and figures guide the reader to an application or result of interest. In addition, they provide an extensive overview on the literature with more than 400 references.
SIAM Review》，第 66 卷，第 3 期，第 401-401 页，2024 年 5 月。 在 "Cardinality Minimization, Constraints, and Regularization：中，Andreas M. Tillmann、Daniel Bienstock、Andrea Lodi 和 Alexandra Schwartz 考虑了一类优化问题，这些问题涉及约束条件或目标函数中变量向量的万有性。这类问题有很多重要应用，例如医学成像（如 X 射线断层扫描）、人脸识别、无线传感器网络设计、选股、晶体学、天文学、计算机视觉、分类和回归、可解释机器学习以及统计数据分析。本文的重点是连续变量，这使它有别于无数经典的运筹学或组合优化问题。本文详细研究了三类一般问题：卡方最小化问题、卡方受限问题和正则化卡方问题。论文提供了一个连接多个学科的路线图，并概述了可用于万有引力优化问题的多种不同计算方法。由于此类问题具有跨学科性质，作者根据具体应用领域组织了综述，并指出了重叠和差异。论文首先介绍了突出的卡方优化问题，即信号和图像处理、投资组合优化和管理、高维统计和机器学习，以及组合学、矩阵稀疏化和组/块稀疏性中的一些相关问题。然后继续介绍精确模型和求解方法。接下来的章节专门讨论了松弛和启发式算法，以及精确算法和启发式算法的可扩展性。作者在组织篇幅较长的论文方面做出了很大努力，这意味着表格和图表可以引导读者找到感兴趣的应用或结果。此外，他们还提供了 400 多篇参考文献，对文献进行了广泛的概述。
{"title":"Survey and Review","authors":"Marlis Hochbruck","doi":"10.1137/24n97592x","DOIUrl":"https://doi.org/10.1137/24n97592x","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 401-401, May 2024. <br/> In “Cardinality Minimization, Constraints, and Regularization: A Survey,\" Andreas M. Tillmann, Daniel Bienstock, Andrea Lodi, and Alexandra Schwartz consider a class of optimization problems that involve the cardinality of variable vectors in constraints or in the objective function. Such problems have many important applications, e.g., medical imaging (like X-ray tomography), face recognition, wireless sensor network design, stock picking, crystallography, astronomy, computer vision, classification and regression, interpretable machine learning, and statistical data analysis. The emphasis in this paper is on continuous variables, which distinguishes it from a myriad of classical operation research or combinatorial optimization problems. Three general problem classes are studied in detail: cardinality minimization problems, cardinality-constrained problems, and regularized cardinality problems. The paper provides a road map connecting several disciplines and offers an overview of many different computational approaches that are available for cardinality optimization problems. Since such problems are of cross-disciplinary nature, the authors organized their review according to specific application areas and point out overlaps and differences. The paper starts with prominent cardinality optimization problems, namely, signal and image processing, portfolio optimization and management, high-dimensional statistics and machine learning, and some related problems from combinatorics, matrix sparsification, and group/block sparsity. It then continues with exact models and solution methods. The further sections are devoted to relaxations and heuristics, scalability of exact and heuristic algorithms. The authors made a strong effort regarding the organization of their quite long paper, meaning that tables and figures guide the reader to an application or result of interest. In addition, they provide an extensive overview on the literature with more than 400 references.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141904644","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Andreas M. Tillmann, Daniel Bienstock, Andrea Lodi, Alexandra Schwartz
SIAM Review, Volume 66, Issue 3, Page 403-477, May 2024. We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and we give concrete examples from diverse application fields such as signal and image processing, portfolio selection, and machine learning. The paper discusses general-purpose modeling techniques and broadly applicable as well as problem-specific exact and heuristic solution approaches. While our perspective is that of mathematical optimization, a main goal of this work is to reach out to and build bridges between the different communities in which cardinality optimization problems are frequently encountered. In particular, we highlight that modern mixed-integer programming, which is often regarded as impractical due to the commonly unsatisfactory behavior of black-box solvers applied to generic problem formulations, can in fact produce provably high-quality or even optimal solutions for cardinality optimization problems, even in large-scale real-world settings. Achieving such performance typically involves drawing on the merits of problem-specific knowledge that may stem from different fields of application and, e.g., can shed light on structural properties of a model or its solutions, or can lead to the development of efficient heuristics. We also provide some illustrative examples.
{"title":"Cardinality Minimization, Constraints, and Regularization: A Survey","authors":"Andreas M. Tillmann, Daniel Bienstock, Andrea Lodi, Alexandra Schwartz","doi":"10.1137/21m142770x","DOIUrl":"https://doi.org/10.1137/21m142770x","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 403-477, May 2024. <br/> We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and we give concrete examples from diverse application fields such as signal and image processing, portfolio selection, and machine learning. The paper discusses general-purpose modeling techniques and broadly applicable as well as problem-specific exact and heuristic solution approaches. While our perspective is that of mathematical optimization, a main goal of this work is to reach out to and build bridges between the different communities in which cardinality optimization problems are frequently encountered. In particular, we highlight that modern mixed-integer programming, which is often regarded as impractical due to the commonly unsatisfactory behavior of black-box solvers applied to generic problem formulations, can in fact produce provably high-quality or even optimal solutions for cardinality optimization problems, even in large-scale real-world settings. Achieving such performance typically involves drawing on the merits of problem-specific knowledge that may stem from different fields of application and, e.g., can shed light on structural properties of a model or its solutions, or can lead to the development of efficient heuristics. We also provide some illustrative examples.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141904540","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 3, Page 533-533, May 2024. The SIGEST article in this issue is “Operator Learning Using Random Features: A Tool for Scientific Computing,” by Nicholas H. Nelsen and Andrew M. Stuart. This work considers the problem of operator learning in infinite-dimensional Banach spaces through the use of random features. The driving application is the approximation of solution operators to partial differential equations (PDEs), here foremost time-dependent problems, that are naturally posed in an infinite-dimensional function space. Typically here, in contrast to the mainstream big data regimes of machine learning applications such as computer vision, high resolution data coming from physical experiments or from computationally expensive simulations of such differential equations is usually small. Fast and approximate surrogates built from such data can be advantageous in building forward models for inverse problems or for doing uncertainty quantification, for instance. Showing how this can be done in infinite dimensions gives rise to approximators which are at the outset resolution and discretization invariant, allowing training on one resolution and deploying on another. At the heart of this work is the function-valued random features methodology that the authors extended from the finite setting of the classical random features approach. Here, the nonlinear operator is approximated by a linear combination of random operators which turn out to be a low-rank approximation and whose computation amounts to a convex, quadratic optimisation problem that is efficiently solvable and for which convergence guarantees can be derived. The methodology is then concretely applied to two concrete PDE examples: Burgers' equations and Darcy flow, demonstrating the applicability of the function-valued random features method, its scalability, discretization invariance, and transferability. The original 2021 article, which appeared in SIAM's Journal on Scientific Computing, has attracted considerable attention. In preparing this SIGEST version, the authors have made numerous modifications and revisions. These include expanding the introductory section and the concluding remarks, condensing the technical content and making it more accessible, and adding a link to an open access GitHub repository that contains all data and code used to produce the results in the paper.
SIAM Review》，第 66 卷第 3 期，第 533-533 页，2024 年 5 月。 本期的 SIGEST 文章是 "Operator Learning Using Random Features：一种科学计算工具"，作者 Nicholas H. Nelsen 和 Andrew M. Stuart。这项研究考虑了通过使用随机特征在无限维巴拿赫空间中进行算子学习的问题。其主要应用是近似偏微分方程（PDEs）的解算子，在这里最重要的是时间相关问题，这些问题自然是在无穷维函数空间中提出的。通常情况下，与计算机视觉等机器学习应用的主流大数据环境不同，来自物理实验或计算成本高昂的此类微分方程模拟的高分辨率数据通常较少。从这些数据中建立快速近似的代用数据，在为逆问题建立前向模型或进行不确定性量化等方面具有优势。通过展示如何在无限维度上实现这一点，我们可以得到从一开始就与分辨率和离散度无关的近似值，从而可以在一种分辨率上进行训练，并在另一种分辨率上进行部署。这项工作的核心是作者从经典随机特征方法的有限设置中扩展出来的函数值随机特征方法。在这里，非线性算子由随机算子的线性组合近似，而随机算子的线性组合是一种低阶近似，其计算相当于一个凸二次优化问题，可高效求解，并可得出收敛保证。然后，我们将这一方法具体应用于两个具体的 PDE 例子：布尔格斯方程和达西流，展示了函数值随机特征方法的适用性、可扩展性、离散不变性和可转移性。最初的 2021 年文章发表在 SIAM 的《科学计算期刊》上，引起了广泛关注。在编写此 SIGEST 版本时，作者进行了大量修改和修订。这些修改和修订包括扩充引言部分和结束语，浓缩技术内容并使其更易于理解，以及添加指向开放访问 GitHub 存储库的链接，该存储库包含用于生成论文结果的所有数据和代码。
{"title":"SIGEST","authors":"The Editors","doi":"10.1137/24n975943","DOIUrl":"https://doi.org/10.1137/24n975943","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 533-533, May 2024. <br/> The SIGEST article in this issue is “Operator Learning Using Random Features: A Tool for Scientific Computing,” by Nicholas H. Nelsen and Andrew M. Stuart. This work considers the problem of operator learning in infinite-dimensional Banach spaces through the use of random features. The driving application is the approximation of solution operators to partial differential equations (PDEs), here foremost time-dependent problems, that are naturally posed in an infinite-dimensional function space. Typically here, in contrast to the mainstream big data regimes of machine learning applications such as computer vision, high resolution data coming from physical experiments or from computationally expensive simulations of such differential equations is usually small. Fast and approximate surrogates built from such data can be advantageous in building forward models for inverse problems or for doing uncertainty quantification, for instance. Showing how this can be done in infinite dimensions gives rise to approximators which are at the outset resolution and discretization invariant, allowing training on one resolution and deploying on another. At the heart of this work is the function-valued random features methodology that the authors extended from the finite setting of the classical random features approach. Here, the nonlinear operator is approximated by a linear combination of random operators which turn out to be a low-rank approximation and whose computation amounts to a convex, quadratic optimisation problem that is efficiently solvable and for which convergence guarantees can be derived. The methodology is then concretely applied to two concrete PDE examples: Burgers' equations and Darcy flow, demonstrating the applicability of the function-valued random features method, its scalability, discretization invariance, and transferability. The original 2021 article, which appeared in SIAM's Journal on Scientific Computing, has attracted considerable attention. In preparing this SIGEST version, the authors have made numerous modifications and revisions. These include expanding the introductory section and the concluding remarks, condensing the technical content and making it more accessible, and adding a link to an open access GitHub repository that contains all data and code used to produce the results in the paper.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141904646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 2, Page 203-203, May 2024. Inverse problems arise in various applications---for instance, in geoscience, biomedical science, or mining engineering, to mention just a few. The purpose is to recover an object or phenomenon from measured data which is typically subject to noise. The article “Computational Methods for Large-Scale Inverse Problems: A Survey on Hybrid Projection Methods,” by Julianne Chung and Silvia Gazzola, focuses on large, mainly linear, inverse problems. The mathematical modeling of such problems results in a linear system with a very large matrix $A in mathbb{R}^{mtimes n}$ and a perturbed right-hand side. In some applications, it is not even possible to store the matrix, and thus algorithms which only use $A$ in the form of matrix-vector products $Ax$ or $A^Tx$ are the only choice. The article starts with two examples from image deblurring and tomographic reconstruction illustrating the challenges of inverse problems. It then presents the basic idea of regularization which consists of augmenting the model by additional information. Two variants of regularization methods are considered in detail, namely, variational and iterative methods. For variational methods it is crucial to know a good regularization parameter in advance. Unfortunately, its estimation can be expensive. On the other hand, iterative schemes, such as Krylov subspace methods, regularize by early termination of the iterations. Hybrid methods combine these two approaches leveraging the best features of each class. The paper focuses on hybrid projection methods. Here, one starts with a Krylov process in which the original problem is projected onto a low-dimensional subspace. The projected problem is then solved using a variational regularization method. The paper reviews the most relevant direct and iterative regularization techniques before it provides details on the two main building blocks of hybrid methods, namely, generating a subspace for the solution and solving the projected problem. It covers theoretical as well as numerical aspects of these schemes and also presents some extensions of hybrid methods: more general Tikhonov problems, nonstandard projection methods (enrichment, augmentation, recycling), $ell_p$ regularization, Bayesian setting, and nonlinear problems. In addition, relevant software packages are provided. The presentation is very clear and the paper is also readable for those who are not experts in the field. Hence, it is valuable for everyone interested in large-scale inverse problems.
{"title":"Survey and Review","authors":"Marlis Hochbruck","doi":"10.1137/24n975876","DOIUrl":"https://doi.org/10.1137/24n975876","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 203-203, May 2024. <br/> Inverse problems arise in various applications---for instance, in geoscience, biomedical science, or mining engineering, to mention just a few. The purpose is to recover an object or phenomenon from measured data which is typically subject to noise. The article “Computational Methods for Large-Scale Inverse Problems: A Survey on Hybrid Projection Methods,” by Julianne Chung and Silvia Gazzola, focuses on large, mainly linear, inverse problems. The mathematical modeling of such problems results in a linear system with a very large matrix $A in mathbb{R}^{mtimes n}$ and a perturbed right-hand side. In some applications, it is not even possible to store the matrix, and thus algorithms which only use $A$ in the form of matrix-vector products $Ax$ or $A^Tx$ are the only choice. The article starts with two examples from image deblurring and tomographic reconstruction illustrating the challenges of inverse problems. It then presents the basic idea of regularization which consists of augmenting the model by additional information. Two variants of regularization methods are considered in detail, namely, variational and iterative methods. For variational methods it is crucial to know a good regularization parameter in advance. Unfortunately, its estimation can be expensive. On the other hand, iterative schemes, such as Krylov subspace methods, regularize by early termination of the iterations. Hybrid methods combine these two approaches leveraging the best features of each class. The paper focuses on hybrid projection methods. Here, one starts with a Krylov process in which the original problem is projected onto a low-dimensional subspace. The projected problem is then solved using a variational regularization method. The paper reviews the most relevant direct and iterative regularization techniques before it provides details on the two main building blocks of hybrid methods, namely, generating a subspace for the solution and solving the projected problem. It covers theoretical as well as numerical aspects of these schemes and also presents some extensions of hybrid methods: more general Tikhonov problems, nonstandard projection methods (enrichment, augmentation, recycling), $ell_p$ regularization, Bayesian setting, and nonlinear problems. In addition, relevant software packages are provided. The presentation is very clear and the paper is also readable for those who are not experts in the field. Hence, it is valuable for everyone interested in large-scale inverse problems.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140902936","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号