SIAM Review, Volume 66, Issue 2, Page 368-387, May 2024. This tutorial describes several basic and much-studied types of interactions with incomplete information, analyzing them by means of evolutionary game dynamics. The games include sender-receiver games, owner-challenger contests, costly advertising, and calls for help. We model the evolution of populations of players reacting to each other and compare adaptive dynamics, replicator dynamics, and best-reply dynamics. In particular, we study signaling norms and nonequilibrium outcomes.
{"title":"Dynamics of Signaling Games","authors":"Hannelore De Silva, Karl Sigmund","doi":"10.1137/23m156402x","DOIUrl":"https://doi.org/10.1137/23m156402x","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 368-387, May 2024. <br/> This tutorial describes several basic and much-studied types of interactions with incomplete information, analyzing them by means of evolutionary game dynamics. The games include sender-receiver games, owner-challenger contests, costly advertising, and calls for help. We model the evolution of populations of players reacting to each other and compare adaptive dynamics, replicator dynamics, and best-reply dynamics. In particular, we study signaling norms and nonequilibrium outcomes.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140903002","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 2, Page 205-284, May 2024. This paper surveys an important class of methods that combine iterative projection methods and variational regularization methods for large-scale inverse problems. Iterative methods such as Krylov subspace methods are invaluable in the numerical linear algebra community and have proved important in solving inverse problems due to their inherent regularizing properties and their ability to handle large-scale problems. Variational regularization describes a broad and important class of methods that are used to obtain reliable solutions to inverse problems, whereby one solves a modified problem that incorporates prior knowledge. Hybrid projection methods combine iterative projection methods with variational regularization techniques in a synergistic way, providing researchers with a powerful computational framework for solving very large inverse problems. Although the idea of a hybrid Krylov method for linear inverse problems goes back to the 1980s, several recent advances on new regularization frameworks and methodologies have made this field ripe for extensions, further analyses, and new applications. In this paper, we provide a practical and accessible introduction to hybrid projection methods in the context of solving large (linear) inverse problems.
{"title":"Computational Methods for Large-Scale Inverse Problems: A Survey on Hybrid Projection Methods","authors":"Julianne Chung, Silvia Gazzola","doi":"10.1137/21m1441420","DOIUrl":"https://doi.org/10.1137/21m1441420","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 205-284, May 2024. <br/> This paper surveys an important class of methods that combine iterative projection methods and variational regularization methods for large-scale inverse problems. Iterative methods such as Krylov subspace methods are invaluable in the numerical linear algebra community and have proved important in solving inverse problems due to their inherent regularizing properties and their ability to handle large-scale problems. Variational regularization describes a broad and important class of methods that are used to obtain reliable solutions to inverse problems, whereby one solves a modified problem that incorporates prior knowledge. Hybrid projection methods combine iterative projection methods with variational regularization techniques in a synergistic way, providing researchers with a powerful computational framework for solving very large inverse problems. Although the idea of a hybrid Krylov method for linear inverse problems goes back to the 1980s, several recent advances on new regularization frameworks and methodologies have made this field ripe for extensions, further analyses, and new applications. In this paper, we provide a practical and accessible introduction to hybrid projection methods in the context of solving large (linear) inverse problems.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140903028","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, Tong Zhang
SIAM Review, Volume 66, Issue 2, Page 319-352, May 2024. We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving this class of problems converge slowly in practice, involve subproblems that can be as difficult as the original problem, or lack rigorous convergence guarantees. In this paper, we propose a manifold proximal gradient method (ManPG) for solving this class of problems. We prove that the proposed method converges globally to a stationary point and establish its iteration complexity for obtaining an $epsilon$-stationary point. Furthermore, we present numerical results on the sparse PCA and compressed modes problems to demonstrate the advantages of the proposed method. We also discuss some recent advances related to ManPG for Riemannian optimization with nonsmooth objective functions.
{"title":"Nonsmooth Optimization over the Stiefel Manifold and Beyond: Proximal Gradient Method and Recent Variants","authors":"Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, Tong Zhang","doi":"10.1137/24m1628578","DOIUrl":"https://doi.org/10.1137/24m1628578","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 319-352, May 2024. <br/> We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving this class of problems converge slowly in practice, involve subproblems that can be as difficult as the original problem, or lack rigorous convergence guarantees. In this paper, we propose a manifold proximal gradient method (ManPG) for solving this class of problems. We prove that the proposed method converges globally to a stationary point and establish its iteration complexity for obtaining an $epsilon$-stationary point. Furthermore, we present numerical results on the sparse PCA and compressed modes problems to demonstrate the advantages of the proposed method. We also discuss some recent advances related to ManPG for Riemannian optimization with nonsmooth objective functions.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140902974","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 2, Page 317-317, May 2024. The SIGEST article in this issue is “Nonsmooth Optimization over the Stiefel Manifold and Beyond: Proximal Gradient Method and Recent Variants,” by Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, and Tong Zhang. This work considers nonsmooth optimization on the Stiefel manifold, the manifold of orthonormal $k$-frames in $mathbb{R}^n$. The authors propose a novel proximal gradient algorithm, coined ManPG, for minimizing the sum of a smooth, potentially nonconvex function, and a convex and potentially nonsmooth function whose arguments live on the Stiefel manifold. In contrast to existing approaches, which either are computationally expensive (due to expensive subproblems or slow convergence) or lack rigorous convergence guarantees, ManPG is thoroughly analyzed and features subproblems that can be computed efficiently. Nonsmooth optimization problems on the Stiefel manifold appear in many applications. In statistics sparse principal component analysis (PCA), that is, PCA that seeks principal components with very few nonzero entries, is a prime example. Unsupervised feature selection (machine learning) and blind deconvolution with a sparsity constraint on the deconvolved signal (inverse problems) are important instances of this general objective structure. At the heart of this work is a beautiful interplay between a theoretically well-founded and efficient novel optimization approach for an important class of problems and a set of computational experiments that demonstrate the effectiveness of this new approach. In order to make proximal gradient work for the Stiefel manifold they add a retraction step to the iterations that keeps the iterates feasible. The authors prove global convergence of ManPG to a stationary point and analyze its computational complexity for approximating the latter to $epsilon$ accuracy. The numerical discussion features results for sparse PCA and the problem of computing compressed modes, that is, spatially localized solutions, of the independent-particle Schrödinger equation. The original 2020 article, which appeared in SIAM Journal on Optimization, has attracted considerable attention. In preparing this SIGEST version, the authors have added a discussion on several subsequent works on algorithms for solving Riemannian optimization with nonsmooth objectives. These works were mostly motivated by the ManPG algorithm and include a manifold proximal point algorithm, manifold proximal linear algorithm, stochastic ManPG, zeroth-order ManPG, Riemannian proximal gradient method, and Riemannian proximal Newton method.
{"title":"SIGEST","authors":"The Editors","doi":"10.1137/24n97589x","DOIUrl":"https://doi.org/10.1137/24n97589x","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 317-317, May 2024. <br/> The SIGEST article in this issue is “Nonsmooth Optimization over the Stiefel Manifold and Beyond: Proximal Gradient Method and Recent Variants,” by Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, and Tong Zhang. This work considers nonsmooth optimization on the Stiefel manifold, the manifold of orthonormal $k$-frames in $mathbb{R}^n$. The authors propose a novel proximal gradient algorithm, coined ManPG, for minimizing the sum of a smooth, potentially nonconvex function, and a convex and potentially nonsmooth function whose arguments live on the Stiefel manifold. In contrast to existing approaches, which either are computationally expensive (due to expensive subproblems or slow convergence) or lack rigorous convergence guarantees, ManPG is thoroughly analyzed and features subproblems that can be computed efficiently. Nonsmooth optimization problems on the Stiefel manifold appear in many applications. In statistics sparse principal component analysis (PCA), that is, PCA that seeks principal components with very few nonzero entries, is a prime example. Unsupervised feature selection (machine learning) and blind deconvolution with a sparsity constraint on the deconvolved signal (inverse problems) are important instances of this general objective structure. At the heart of this work is a beautiful interplay between a theoretically well-founded and efficient novel optimization approach for an important class of problems and a set of computational experiments that demonstrate the effectiveness of this new approach. In order to make proximal gradient work for the Stiefel manifold they add a retraction step to the iterations that keeps the iterates feasible. The authors prove global convergence of ManPG to a stationary point and analyze its computational complexity for approximating the latter to $epsilon$ accuracy. The numerical discussion features results for sparse PCA and the problem of computing compressed modes, that is, spatially localized solutions, of the independent-particle Schrödinger equation. The original 2020 article, which appeared in SIAM Journal on Optimization, has attracted considerable attention. In preparing this SIGEST version, the authors have added a discussion on several subsequent works on algorithms for solving Riemannian optimization with nonsmooth objectives. These works were mostly motivated by the ManPG algorithm and include a manifold proximal point algorithm, manifold proximal linear algorithm, stochastic ManPG, zeroth-order ManPG, Riemannian proximal gradient method, and Riemannian proximal Newton method.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140903048","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Leslie F. Greengard, Shidong Jiang, Manas Rachh, Jun Wang
SIAM Review, Volume 66, Issue 2, Page 287-315, May 2024. We present a new version of the fast Gauss transform (FGT) for discrete and continuous sources. Classical Hermite expansions are avoided entirely, making use only of the plane-wave representation of the Gaussian kernel and a new hierarchical merging scheme. For continuous source distributions sampled on adaptive tensor product grids, we exploit the separable structure of the Gaussian kernel to accelerate the computation. For discrete sources, the scheme relies on the nonuniform fast Fourier transform (NUFFT) to construct near field plane-wave representations. The scheme has been implemented for either free-space or periodic boundary conditions. In many regimes, the speed is comparable to or better than that of the conventional FFT in work per grid point, despite being fully adaptive.
{"title":"A New Version of the Adaptive Fast Gauss Transform for Discrete and Continuous Sources","authors":"Leslie F. Greengard, Shidong Jiang, Manas Rachh, Jun Wang","doi":"10.1137/23m1572453","DOIUrl":"https://doi.org/10.1137/23m1572453","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 287-315, May 2024. <br/> We present a new version of the fast Gauss transform (FGT) for discrete and continuous sources. Classical Hermite expansions are avoided entirely, making use only of the plane-wave representation of the Gaussian kernel and a new hierarchical merging scheme. For continuous source distributions sampled on adaptive tensor product grids, we exploit the separable structure of the Gaussian kernel to accelerate the computation. For discrete sources, the scheme relies on the nonuniform fast Fourier transform (NUFFT) to construct near field plane-wave representations. The scheme has been implemented for either free-space or periodic boundary conditions. In many regimes, the speed is comparable to or better than that of the conventional FFT in work per grid point, despite being fully adaptive.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140902918","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 2, Page 353-353, May 2024. In this issue the Education section presents two contributions. The first paper, “The Poincaré Metric and the Bergman Theory,” by Steven G. Krantz, discusses the Poincaré metric on the unit disc in the complex space and the Bergman metric on an arbitrary domain in any dimensional complex space. To define the Bergman metric the notion of Bergman kernel is crucial. Some striking properties of the Bergman kernel are discussed briefly, and it is calculated when the domain is the open unit ball. The Bergman metric is invariant under biholomorphic maps. The paper ends by discussing several attractive applications. To incorporate invariance within models in applied science, in particular for machine learning applications, there is currently a considerable interest in non-Euclidean metrics, in invariant (under some actions) metrics, and in reproducing kernels, mostly in the real-valued framework. The Bergman theory (1921) is a special case of Aronszajn's theory of Hilbert spaces with reproducing kernels (1950). Invariant metrics are used, in particular, in the study of partial differential equations. Complex-valued kernels have some interesting connections to linear systems theory. This article sheds some new light on the Poincaré metric, the Bergman kernel, the Bergman metric, and their applications in a manner that helps the reader become accustomed to these notions and to enjoy their properties. The second paper, “Dynamics of Signaling Games,” is presented by Hannelore De Silva and Karl Sigmund and is devoted to much-studied types of interactions with incomplete information, analyzing them by means of evolutionary game dynamics. Game theory is often encountered in models describing economic, social, and biological behavior, where decisions can not only be shaped by rational arguments, but may also be influenced by other factors and players. However, it is often restricted to an analysis of equilibria. In signaling games some agents are less informed than others and try to deal with it by observing actions (signals) from better informed agents. Such signals may be even purposely wrong. This article offers a concise guided tour of outcomes of evolutionary dynamics in a number of small dimensional signaling games focusing on the replicator dynamics, the best-reply dynamics, and the adaptive dynamics (dynamics of behavioral strategies whose vector field follows the gradient of the payoff vector). Furthermore, for the model of evolution of populations of players, the authors compare these dynamics. Several interesting examples illustrate that even simple adaptation processes can lead to nonequilibrium outcomes and endless cycling. This tutorial is targeted at graduate/Ph.D. students and researchers who know the basics of game theory and want to learn examples of signaling games, together with evolutionary game theory.
SIAM 评论》，第 66 卷第 2 期，第 353-353 页，2024 年 5 月。 本期教育版块刊登了两篇论文。第一篇论文是 Steven G. Krantz 撰写的 "The Poincaré Metric and the Bergman Theory"，讨论了复数空间中单位圆盘上的 Poincaré 度量和任意维复数空间中任意域上的 Bergman 度量。要定义伯格曼度量，伯格曼核的概念至关重要。本文简要讨论了伯格曼核的一些显著性质，并计算了当域为开放单位球时的伯格曼核。伯格曼度量在双全形映射下是不变的。论文最后讨论了几个有吸引力的应用。为了将不变性纳入应用科学模型，特别是机器学习应用，目前人们对非欧几里得度量、不变性（在某些作用下）度量和再现核（主要在实值框架内）相当感兴趣。伯格曼理论（1921 年）是阿隆札恩的重现核希尔伯特空间理论（1950 年）的一个特例。不变度量尤其用于偏微分方程的研究。复值核与线性系统理论有一些有趣的联系。这篇文章对庞加莱度量、伯格曼核、伯格曼度量及其应用作了一些新的阐释，有助于读者习惯这些概念并享受它们的特性。第二篇论文题为 "信号博弈动力学"，由汉内洛尔-德-席尔瓦和卡尔-西格蒙德（Karl Sigmund）撰写，专门讨论不完全信息下备受研究的互动类型，并通过演化博弈动力学对其进行分析。博弈论经常出现在描述经济、社会和生物行为的模型中，在这些模型中，决策不仅受理性论证的影响，还可能受其他因素和参与者的影响。然而，博弈论往往局限于对均衡状态的分析。在信号博弈中，一些行为主体的信息不如其他行为主体灵通，他们会试图通过观察信息更灵通的行为主体的行动（信号）来解决这个问题。这些信号甚至可能是故意错误的。本文简要介绍了一些小维度信号博弈中的演化动力学结果，重点关注复制者动力学、最佳回应动力学和适应性动力学（其向量场跟随报酬向量梯度的行为策略动力学）。此外，作者还针对玩家群体的进化模型，对这些动力学进行了比较。几个有趣的例子说明，即使是简单的适应过程也会导致非均衡结果和无休止的循环。本教程面向了解博弈论基础知识并希望结合进化博弈论学习信号博弈实例的研究生/博士生和研究人员。
{"title":"Education","authors":"Hélène Frankowska","doi":"10.1137/24n975906","DOIUrl":"https://doi.org/10.1137/24n975906","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 353-353, May 2024. <br/> In this issue the Education section presents two contributions. The first paper, “The Poincaré Metric and the Bergman Theory,” by Steven G. Krantz, discusses the Poincaré metric on the unit disc in the complex space and the Bergman metric on an arbitrary domain in any dimensional complex space. To define the Bergman metric the notion of Bergman kernel is crucial. Some striking properties of the Bergman kernel are discussed briefly, and it is calculated when the domain is the open unit ball. The Bergman metric is invariant under biholomorphic maps. The paper ends by discussing several attractive applications. To incorporate invariance within models in applied science, in particular for machine learning applications, there is currently a considerable interest in non-Euclidean metrics, in invariant (under some actions) metrics, and in reproducing kernels, mostly in the real-valued framework. The Bergman theory (1921) is a special case of Aronszajn's theory of Hilbert spaces with reproducing kernels (1950). Invariant metrics are used, in particular, in the study of partial differential equations. Complex-valued kernels have some interesting connections to linear systems theory. This article sheds some new light on the Poincaré metric, the Bergman kernel, the Bergman metric, and their applications in a manner that helps the reader become accustomed to these notions and to enjoy their properties. The second paper, “Dynamics of Signaling Games,” is presented by Hannelore De Silva and Karl Sigmund and is devoted to much-studied types of interactions with incomplete information, analyzing them by means of evolutionary game dynamics. Game theory is often encountered in models describing economic, social, and biological behavior, where decisions can not only be shaped by rational arguments, but may also be influenced by other factors and players. However, it is often restricted to an analysis of equilibria. In signaling games some agents are less informed than others and try to deal with it by observing actions (signals) from better informed agents. Such signals may be even purposely wrong. This article offers a concise guided tour of outcomes of evolutionary dynamics in a number of small dimensional signaling games focusing on the replicator dynamics, the best-reply dynamics, and the adaptive dynamics (dynamics of behavioral strategies whose vector field follows the gradient of the payoff vector). Furthermore, for the model of evolution of populations of players, the authors compare these dynamics. Several interesting examples illustrate that even simple adaptation processes can lead to nonequilibrium outcomes and endless cycling. This tutorial is targeted at graduate/Ph.D. students and researchers who know the basics of game theory and want to learn examples of signaling games, together with evolutionary game theory.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140902933","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 2, Page 355-367, May 2024. We treat the Poincaré metric on the disc. In particular we emphasize the fact that it is the canonical holomorphically invariant metric on the unit disc. Then we generalize these ideas to the Bergman metric on a domain in complex space. Along the way we treat the Bergman kernel and study its invariance and uniqueness properties.
{"title":"The Poincaré Metric and the Bergman Theory","authors":"Steven G. Krantz","doi":"10.1137/22m1544622","DOIUrl":"https://doi.org/10.1137/22m1544622","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 355-367, May 2024. <br/> We treat the Poincaré metric on the disc. In particular we emphasize the fact that it is the canonical holomorphically invariant metric on the unit disc. Then we generalize these ideas to the Bergman metric on a domain in complex space. Along the way we treat the Bergman kernel and study its invariance and uniqueness properties.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140902896","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 2, Page 285-285, May 2024. The Gauss transform---convolution with a Gaussian in the continuous case and the sum of $N$ Gaussians at $M$ points in the discrete case---is ubiquitous in applied mathematics, from solving ordinary and partial differential equations to probability density estimation to science applications in astrophysics, image processing, quantum mechanics, and beyond. For the discrete case, the fast Gauss transform (FGT) enables the approximate calculation of the sum of $N$ Gaussians at $M$ points in order $N + M$ (instead of $NM$) operations by a fast summation strategy, which shares work between the sums at different points, similarly to the fast multipole method. In this issue's Research Spotlights section, “A New Version of the Adaptive Fast Gauss Transform for Discrete and Continuous Sources,” authors Leslie F. Greengard, Shidong Jiang, Manas Rachh, and Jun Wang present a new FGT technique that avoids the use of Hermite and local expansions. The new technique employs Fourier spectral approximations, which are accelerated by nonuniform fast Fourier transforms, and results in a considerably more efficient adaptive implementation. Adaptivity is especially vital for realizing the acceleration from a fast transform when points are highly nonuniform. The paper presents compelling illustrations and examples of the computational approach and the adaptive tree-based hierarchy employed. This hierarchy is used to resolve point distributions down to a refinement level determined by accuracy demands; this results in significantly better work per grid point than conventional FGT techniques. Consequently, the authors note that there are potential key benefits in parallelization of the proposed technique. In addition to the technique's clever composition of a broad variety of advanced computing paradigms and exploitation of mathematical structure to facilitate such fast transforms, the authors present several pathways of future research. For example, the analysis is readily accessible from dimensions larger than the illustrative examples illuminate, and univariate sum-of-exponentials structure also may be exploited; the computing techniques detailed by the authors could be tailored to such regimes. These future directions have broad application in scientific computing.
{"title":"Research Spotlights","authors":"Stefan M. Wild","doi":"10.1137/24n975888","DOIUrl":"https://doi.org/10.1137/24n975888","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 285-285, May 2024. <br/> The Gauss transform---convolution with a Gaussian in the continuous case and the sum of $N$ Gaussians at $M$ points in the discrete case---is ubiquitous in applied mathematics, from solving ordinary and partial differential equations to probability density estimation to science applications in astrophysics, image processing, quantum mechanics, and beyond. For the discrete case, the fast Gauss transform (FGT) enables the approximate calculation of the sum of $N$ Gaussians at $M$ points in order $N + M$ (instead of $NM$) operations by a fast summation strategy, which shares work between the sums at different points, similarly to the fast multipole method. In this issue's Research Spotlights section, “A New Version of the Adaptive Fast Gauss Transform for Discrete and Continuous Sources,” authors Leslie F. Greengard, Shidong Jiang, Manas Rachh, and Jun Wang present a new FGT technique that avoids the use of Hermite and local expansions. The new technique employs Fourier spectral approximations, which are accelerated by nonuniform fast Fourier transforms, and results in a considerably more efficient adaptive implementation. Adaptivity is especially vital for realizing the acceleration from a fast transform when points are highly nonuniform. The paper presents compelling illustrations and examples of the computational approach and the adaptive tree-based hierarchy employed. This hierarchy is used to resolve point distributions down to a refinement level determined by accuracy demands; this results in significantly better work per grid point than conventional FGT techniques. Consequently, the authors note that there are potential key benefits in parallelization of the proposed technique. In addition to the technique's clever composition of a broad variety of advanced computing paradigms and exploitation of mathematical structure to facilitate such fast transforms, the authors present several pathways of future research. For example, the analysis is readily accessible from dimensions larger than the illustrative examples illuminate, and univariate sum-of-exponentials structure also may be exploited; the computing techniques detailed by the authors could be tailored to such regimes. These future directions have broad application in scientific computing.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140903007","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zongren Zou, Xuhui Meng, Apostolos F. Psaros, George E. Karniadakis
SIAM Review, Volume 66, Issue 1, Page 161-190, February 2024. Uncertainty quantification (UQ) in machine learning is currently drawing increasing research interest, driven by the rapid deployment of deep neural networks across different fields, such as computer vision and natural language processing, and by the need for reliable tools in risk-sensitive applications. Recently, various machine learning models have also been developed to tackle problems in the field of scientific computing with applications to computational science and engineering (CSE). Physics-informed neural networks and deep operator networks are two such models for solving partial differential equations (PDEs) and learning operator mappings, respectively. In this regard, a comprehensive study of UQ methods tailored specifically for scientific machine learning (SciML) models has been provided in [A. F. Psaros et al., J. Comput. Phys., 477 (2023), art. 111902]. Nevertheless, and despite their theoretical merit, implementations of these methods are not straightforward, especially in large-scale CSE applications, hindering their broad adoption in both research and industry settings. In this paper, we present an open-source Python library (ŭlhttps://github.com/Crunch-UQ4MI), termed NeuralUQ and accompanied by an educational tutorial, for employing UQ methods for SciML in a convenient and structured manner. The library, designed for both educational and research purposes, supports multiple modern UQ methods and SciML models. It is based on a succinct workflow and facilitates flexible employment and easy extensions by the users. We first present a tutorial of NeuralUQ and subsequently demonstrate its applicability and efficiency in four diverse examples, involving dynamical systems and high-dimensional parametric and time-dependent PDEs.
{"title":"NeuralUQ: A Comprehensive Library for Uncertainty Quantification in Neural Differential Equations and Operators","authors":"Zongren Zou, Xuhui Meng, Apostolos F. Psaros, George E. Karniadakis","doi":"10.1137/22m1518189","DOIUrl":"https://doi.org/10.1137/22m1518189","url":null,"abstract":"SIAM Review, Volume 66, Issue 1, Page 161-190, February 2024. <br/> Uncertainty quantification (UQ) in machine learning is currently drawing increasing research interest, driven by the rapid deployment of deep neural networks across different fields, such as computer vision and natural language processing, and by the need for reliable tools in risk-sensitive applications. Recently, various machine learning models have also been developed to tackle problems in the field of scientific computing with applications to computational science and engineering (CSE). Physics-informed neural networks and deep operator networks are two such models for solving partial differential equations (PDEs) and learning operator mappings, respectively. In this regard, a comprehensive study of UQ methods tailored specifically for scientific machine learning (SciML) models has been provided in [A. F. Psaros et al., J. Comput. Phys., 477 (2023), art. 111902]. Nevertheless, and despite their theoretical merit, implementations of these methods are not straightforward, especially in large-scale CSE applications, hindering their broad adoption in both research and industry settings. In this paper, we present an open-source Python library (ŭlhttps://github.com/Crunch-UQ4MI), termed NeuralUQ and accompanied by an educational tutorial, for employing UQ methods for SciML in a convenient and structured manner. The library, designed for both educational and research purposes, supports multiple modern UQ methods and SciML models. It is based on a succinct workflow and facilitates flexible employment and easy extensions by the users. We first present a tutorial of NeuralUQ and subsequently demonstrate its applicability and efficiency in four diverse examples, involving dynamical systems and high-dimensional parametric and time-dependent PDEs.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139705027","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 66, Issue 1, Page 147-147, February 2024. In this issue the Education section presents two contributions. The first paper, “Resonantly Forced ODEs and Repeated Roots,” is written by Allan R. Willms. The resonant forcing problem is as follows: find $y(cdot)$ such that $L[y(x)]=u(x)$, where $L[u(x)]=0$ and $L=a_0(x) + sum_{j=1}^n a_j(x) frac{d^j}{dx^j}$. The repeated roots problem consists in finding $mn$ linearly independent solutions to $L^m[y(x)]=0$ under the assumption that $n$ linearly independent solutions to $L[y(x)]= 0$ are known. A recent article by B. Gouveia and H. A. Stone, “Generating Resonant and Repeated Root Solutions to Ordinary Differential Equations Using Perturbation Methods” [SIAM Rev., 64 (2022), pp. 485--499], discusses a method for finding solutions to these two problems. This new contribution observes that by applying the same mathematical justifications, one may get similar results in a simpler way. The starting point consists in defining operators $L_lambda := hat L -g(lambda)$ with $L_{lambda_0}=L$ for some $lambda_0$ and of a parameter-dependent family of solutions to the homogeneous equations $L_lambda[y(x;lambda)]=0$. Under appropriate assumptions on $g$, differentiating this equality allows one to get solutions to problems of interest. This approach is illustrated on nine examples, seven of which are the same as in the publication of B. Gouveia and H. A. Stone, where for each example $g$ and $hat L$ are appropriately chosen. This approach may be included in a course of ordinary differential equations (ODEs) as a methodology for finding solutions to these two particular classes of ODEs. It can also be used by undergraduate students for individual training as an alternative to variation of parameters. The second paper, “NeuralUQ: A Comprehensive Library for Uncertainty Quantification in Neural Differential Equations and Operators,” is presented by Zongren Zou, Xuhui Meng, Apostolos Psaros, and George E. Karniadakis. In machine learning uncertainty quantification (UQ) is a hot research topic, driven by various questions arising in computer vision and natural language processing, and by risk-sensitive applications. Numerous machine learning models, such as, for instance, physics-informed neural networks and deep operator networks, help in solving partial differential equations and learning operator mappings, respectively. However, some data may be noisy and/or sampled at random locations. This paper presents an open-source Python library (https://github.com/Crunch-UQ4MI) for employing a reliable toolbox of UQ methods for scientific machine learning. It is designed for both educational and research purposes and is illustrated on four examples, involving dynamical systems and high-dimensional parametric and time-dependent PDEs. NeuralUQ is planned to be constantly updated.
SIAM 评论》，第 66 卷第 1 期，第 147-147 页，2024 年 2 月。 本期教育版块刊登了两篇论文。第一篇论文题为 "共振强迫 ODEs 和重复根"，作者是 Allan R. Willms。共振强迫问题如下：求 $y(cdot)$ 使 $L[y(x)]=u(x)$，其中 $L[u(x)]=0$ 和 $L=a_0(x)+sum_{j=1}^n a_j(x) frac{d^j}{dx^j}$。重复根问题包括在已知 $n$ 线性独立解 $L[y(x)]=0$ 的前提下，找到 $mn$ 线性独立解 $L^m[y(x)]=0$。B. Gouveia 和 H. A. Stone 最近发表的一篇文章 "使用扰动方法生成常微分方程的共振解和重复根解" [SIAM Rev., 64 (2022), pp.这篇新论文指出，通过应用相同的数学原理，我们可以用更简单的方法得到类似的结果。出发点包括定义算子 $L_lambda := hat L -g(lambda)$，其中 $L_{lambda_0}=L 为某个 $lambda_0$，以及同质方程 $L_lambda[y(x;lambda)]=0$的解的参数依赖族。在对 $g$ 作适当假设的情况下，微分这个等式就能得到相关问题的解。我们用九个例子来说明这种方法，其中七个与 B. Gouveia 和 H. A. Stone 出版物中的例子相同，每个例子中的 $g$ 和 $hat L$ 都经过适当选择。这种方法可以作为寻找这两类特殊 ODE 的解的方法纳入常微分方程（ODE）课程。本科生也可以用这种方法进行个人训练，作为参数变化的替代方法。第二篇论文题为 "NeuralUQ：神经微分方程和算子中不确定性量化的综合库"，由邹宗仁、孟旭辉、Apostolos Psaros 和 George E. Karniadakis 发表。在机器学习领域，不确定性量化（UQ）是一个热门研究课题，由计算机视觉和自然语言处理中出现的各种问题以及对风险敏感的应用所驱动。许多机器学习模型，例如物理信息神经网络和深度算子网络，分别有助于求解偏微分方程和学习算子映射。然而，有些数据可能存在噪声和/或采样位置随机。本文介绍了一个开源 Python 库（https://github.com/Crunch-UQ4MI），用于在科学机器学习中使用可靠的 UQ 方法工具箱。该库专为教育和研究目的而设计，并通过四个例子进行了说明，涉及动力系统和高维参数与时间相关的 PDE。NeuralUQ 计划不断更新。
{"title":"Education","authors":"Helene Frankowska","doi":"10.1137/24n975852","DOIUrl":"https://doi.org/10.1137/24n975852","url":null,"abstract":"SIAM Review, Volume 66, Issue 1, Page 147-147, February 2024. <br/> In this issue the Education section presents two contributions. The first paper, “Resonantly Forced ODEs and Repeated Roots,” is written by Allan R. Willms. The resonant forcing problem is as follows: find $y(cdot)$ such that $L[y(x)]=u(x)$, where $L[u(x)]=0$ and $L=a_0(x) + sum_{j=1}^n a_j(x) frac{d^j}{dx^j}$. The repeated roots problem consists in finding $mn$ linearly independent solutions to $L^m[y(x)]=0$ under the assumption that $n$ linearly independent solutions to $L[y(x)]= 0$ are known. A recent article by B. Gouveia and H. A. Stone, “Generating Resonant and Repeated Root Solutions to Ordinary Differential Equations Using Perturbation Methods” [SIAM Rev., 64 (2022), pp. 485--499], discusses a method for finding solutions to these two problems. This new contribution observes that by applying the same mathematical justifications, one may get similar results in a simpler way. The starting point consists in defining operators $L_lambda := hat L -g(lambda)$ with $L_{lambda_0}=L$ for some $lambda_0$ and of a parameter-dependent family of solutions to the homogeneous equations $L_lambda[y(x;lambda)]=0$. Under appropriate assumptions on $g$, differentiating this equality allows one to get solutions to problems of interest. This approach is illustrated on nine examples, seven of which are the same as in the publication of B. Gouveia and H. A. Stone, where for each example $g$ and $hat L$ are appropriately chosen. This approach may be included in a course of ordinary differential equations (ODEs) as a methodology for finding solutions to these two particular classes of ODEs. It can also be used by undergraduate students for individual training as an alternative to variation of parameters. The second paper, “NeuralUQ: A Comprehensive Library for Uncertainty Quantification in Neural Differential Equations and Operators,” is presented by Zongren Zou, Xuhui Meng, Apostolos Psaros, and George E. Karniadakis. In machine learning uncertainty quantification (UQ) is a hot research topic, driven by various questions arising in computer vision and natural language processing, and by risk-sensitive applications. Numerous machine learning models, such as, for instance, physics-informed neural networks and deep operator networks, help in solving partial differential equations and learning operator mappings, respectively. However, some data may be noisy and/or sampled at random locations. This paper presents an open-source Python library (https://github.com/Crunch-UQ4MI) for employing a reliable toolbox of UQ methods for scientific machine learning. It is designed for both educational and research purposes and is illustrated on four examples, involving dynamical systems and high-dimensional parametric and time-dependent PDEs. NeuralUQ is planned to be constantly updated.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2024-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139705088","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号