SIAM Review, Volume 65, Issue 3, Page 735-773, August 2023. We analyze neural ordinary differential equations (NODEs) from a control theoretical perspective to address some of the main properties and paradigms of deep learning (DL), in particular, data classification and universal approximation. These objectives are tackled and achieved from the perspective of the simultaneous control of systems of NODEs. For instance, in the context of classification, each item to be classified corresponds to a different initial datum for the control problem of the NODE, to be classified, all of them by the same common control, to the location (a subdomain of the Euclidean space) associated to each label. Our proofs are genuinely nonlinear and constructive, allowing us to estimate the complexity of the control strategies we develop. The nonlinear nature of the activation functions governing the dynamics of NODEs under consideration plays a key role in our proofs, since it allows deforming half of the phase space while the other half remains invariant, a property that classical models in mechanics do not fulfill. This very property allows us to build elementary controls inducing specific dynamics and transformations whose concatenation, along with properly chosen hyperplanes, allows us to achieve our goals in finitely many steps. The nonlinearity of the dynamics is assumed to be Lipschitz. Therefore, our results apply also in the particular case of the ReLU activation function. We also present the counterparts in the context of the control of neural transport equations, establishing a link between optimal transport and deep neural networks.
{"title":"Neural ODE Control for Classification, Approximation, and Transport","authors":"Domènec Ruiz-Balet, Enrique Zuazua","doi":"10.1137/21m1411433","DOIUrl":"https://doi.org/10.1137/21m1411433","url":null,"abstract":"SIAM Review, Volume 65, Issue 3, Page 735-773, August 2023. <br/> We analyze neural ordinary differential equations (NODEs) from a control theoretical perspective to address some of the main properties and paradigms of deep learning (DL), in particular, data classification and universal approximation. These objectives are tackled and achieved from the perspective of the simultaneous control of systems of NODEs. For instance, in the context of classification, each item to be classified corresponds to a different initial datum for the control problem of the NODE, to be classified, all of them by the same common control, to the location (a subdomain of the Euclidean space) associated to each label. Our proofs are genuinely nonlinear and constructive, allowing us to estimate the complexity of the control strategies we develop. The nonlinear nature of the activation functions governing the dynamics of NODEs under consideration plays a key role in our proofs, since it allows deforming half of the phase space while the other half remains invariant, a property that classical models in mechanics do not fulfill. This very property allows us to build elementary controls inducing specific dynamics and transformations whose concatenation, along with properly chosen hyperplanes, allows us to achieve our goals in finitely many steps. The nonlinearity of the dynamics is assumed to be Lipschitz. Therefore, our results apply also in the particular case of the ReLU activation function. We also present the counterparts in the context of the control of neural transport equations, establishing a link between optimal transport and deep neural networks.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2023-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71518431","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 65, Issue 3, Page 774-805, August 2023. The beauty and simplicity of compartment modeling makes it a useful approach for simulating dynamics in an amazingly wide range of applications, which are growing rapidly especially in global carbon cycling, hydrological network flows, and epidemiology and population dynamics. These contexts, however, often involve compartment-to-compartment flows that are incongruent with the conventional assumption of complete mixing that results in exponential residence times in linear models. Here we detail a general method for assigning any desired residence time distribution to a given intercompartmental flow, extending compartment modeling capability to transport operations, power-law residence times, diffusions, etc., without resorting to composite compartments, fractional calculus, or partial differential equations (PDEs) for diffusive transport. This is achieved by writing the mass exchange rate coefficients as functions of age-in-compartment as done in one of the first compartment models in 1917, at the cost of converting the usual ordinary differential equations to a system of first-order PDEs. The PDEs are readily converted to a system of integral equations for which a numerical method is devised. Example calculations demonstrate incorporation of advective lags, advective-dispersive transport, power-law residence time distributions, or diffusive domains in compartment models.
{"title":"Compartment Models with Memory","authors":"Timothy Ginn, Lynn Schreyer","doi":"10.1137/21m1437160","DOIUrl":"https://doi.org/10.1137/21m1437160","url":null,"abstract":"SIAM Review, Volume 65, Issue 3, Page 774-805, August 2023. <br/> The beauty and simplicity of compartment modeling makes it a useful approach for simulating dynamics in an amazingly wide range of applications, which are growing rapidly especially in global carbon cycling, hydrological network flows, and epidemiology and population dynamics. These contexts, however, often involve compartment-to-compartment flows that are incongruent with the conventional assumption of complete mixing that results in exponential residence times in linear models. Here we detail a general method for assigning any desired residence time distribution to a given intercompartmental flow, extending compartment modeling capability to transport operations, power-law residence times, diffusions, etc., without resorting to composite compartments, fractional calculus, or partial differential equations (PDEs) for diffusive transport. This is achieved by writing the mass exchange rate coefficients as functions of age-in-compartment as done in one of the first compartment models in 1917, at the cost of converting the usual ordinary differential equations to a system of first-order PDEs. The PDEs are readily converted to a system of integral equations for which a numerical method is devised. Example calculations demonstrate incorporation of advective lags, advective-dispersive transport, power-law residence time distributions, or diffusive domains in compartment models.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2023-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71509717","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 65, Issue 3, Page 732-732, August 2023. This erratum corrects an error in the coefficients of equation (6.23) in the original paper [H. Miao, X. Xia, A. S. Perelson, and H. Wu, SIAM Rev., 53 (2011), pp. 3--39].
{"title":"Erratum: On Identifiability of Nonlinear ODE Models and Applications in Viral Dynamics","authors":"Hongyu Miao, Alan S. Perelson, Hulin Wu","doi":"10.1137/23m1568958","DOIUrl":"https://doi.org/10.1137/23m1568958","url":null,"abstract":"SIAM Review, Volume 65, Issue 3, Page 732-732, August 2023. <br/> This erratum corrects an error in the coefficients of equation (6.23) in the original paper [H. Miao, X. Xia, A. S. Perelson, and H. Wu, SIAM Rev., 53 (2011), pp. 3--39].","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2023-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71509715","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 65, Issue 3, Page 887-902, August 2023. Some simple models of a child swinging on a playground swing are presented. These are analyzed using techniques from Lagrangian mechanics with a twist: the child changes the configuration of the system by sudden movements of their body at key moments in the oscillation. This can lead to jumps in the generalized coordinates describing the system and/or their velocities. Jump conditions can be determined by integrating the Euler--Lagrange equations over a short time interval and then taking the limit as this time interval goes to zero. These models give insights into strategies used by swingers, and answer such vexed questions such as whether it is possible for the swing to go through a full 360$^circ$ turn over its pivot. A model of an instability at the pivot observed by Colin Furze in a rigid swing constructed to rotate through 360$^circ$ is also described. This uses a novel double pendulum configuration in which the two components of the pendulums are constrained to move in orthogonal planes.
{"title":"Piecewise Smooth Models of Pumping a Child's Swing","authors":"Brigid Murphy, Paul Glendinning","doi":"10.1137/19m1268574","DOIUrl":"https://doi.org/10.1137/19m1268574","url":null,"abstract":"SIAM Review, Volume 65, Issue 3, Page 887-902, August 2023. <br/> Some simple models of a child swinging on a playground swing are presented. These are analyzed using techniques from Lagrangian mechanics with a twist: the child changes the configuration of the system by sudden movements of their body at key moments in the oscillation. This can lead to jumps in the generalized coordinates describing the system and/or their velocities. Jump conditions can be determined by integrating the Euler--Lagrange equations over a short time interval and then taking the limit as this time interval goes to zero. These models give insights into strategies used by swingers, and answer such vexed questions such as whether it is possible for the swing to go through a full 360$^circ$ turn over its pivot. A model of an instability at the pivot observed by Colin Furze in a rigid swing constructed to rotate through 360$^circ$ is also described. This uses a novel double pendulum configuration in which the two components of the pendulums are constrained to move in orthogonal planes.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2023-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71516803","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 65, Issue 3, Page 599-599, August 2023. Apart from a short erratum, which concerns the correction of some coefficients in a differential equation in the original paper, this issue contains two Survey and Review articles. “On and Beyond Total Variation Regularization in Imaging: The Role of Space Variance,” authored by Monica Pragliola, Luca Calatroni, Alessandro Lanza, and Fiorella Sgallari, reviews total variation (TV)-type image reconstruction algorithms with a focus on Bayesian interpretations. The paper scientifically travels across various disciplines by considering a standard example problem to highlight extensions for the TV regularization model. A main contribution is a space-variant framework which allows one to describe the contents of an image at a local scale. Important applications of space-variant models are tomography, e.g., magnetic resonance imaging, electrical impedance tomography, positron emission tomography, and photoacoustic tomography, or noninvasive digital reconstruction, e.g., for ancient frescoes, illuminated manuscripts, surface colorization, etc. The unified view of many of the different models within the Bayesian framework enables one to design flexible and adaptive image regularization functionals which take advantage of the form of the underlying gradient distributions through statistical approaches. The paper contains theoretical results as well as sections on algorithmic optimization (based on the alternating direction methods of multipliers) and numerical tests for examples from image deblurring. Thus it should be interesting for researchers from several disciplines. “What Are Higher-Order Networks” is a question raised and answered by Christian Bick, Elizabeth Gross, Heather A. Harrington, and Michael T. Schaub. In short, higher-order networks are a refurbishment of graphs, removing/overcoming some of the limitations of pairwise relationships by enabling the modeling of polyadic relations in real-world systems, such as reactions in biochemical systems with several species or reagents, or interactions of multiple people in social networks. The main topics of discussion are the understanding of the “shape” of data (by identifying and classifying topological and geometrical properties of the data), the modeling of relational data via higher-order networks, and network dynamical systems (describing couplings between dynamical units). The focus of the presentation is on the mathematical aspects of the topics, but a multitude of applications are mentioned. The impressive list of references comprises 316 entries. We believe the paper to be interesting for a broad audience.
{"title":"Survey and Review","authors":"Marlis Hochbruck","doi":"10.1137/23n975727","DOIUrl":"https://doi.org/10.1137/23n975727","url":null,"abstract":"SIAM Review, Volume 65, Issue 3, Page 599-599, August 2023. <br/> Apart from a short erratum, which concerns the correction of some coefficients in a differential equation in the original paper, this issue contains two Survey and Review articles. “On and Beyond Total Variation Regularization in Imaging: The Role of Space Variance,” authored by Monica Pragliola, Luca Calatroni, Alessandro Lanza, and Fiorella Sgallari, reviews total variation (TV)-type image reconstruction algorithms with a focus on Bayesian interpretations. The paper scientifically travels across various disciplines by considering a standard example problem to highlight extensions for the TV regularization model. A main contribution is a space-variant framework which allows one to describe the contents of an image at a local scale. Important applications of space-variant models are tomography, e.g., magnetic resonance imaging, electrical impedance tomography, positron emission tomography, and photoacoustic tomography, or noninvasive digital reconstruction, e.g., for ancient frescoes, illuminated manuscripts, surface colorization, etc. The unified view of many of the different models within the Bayesian framework enables one to design flexible and adaptive image regularization functionals which take advantage of the form of the underlying gradient distributions through statistical approaches. The paper contains theoretical results as well as sections on algorithmic optimization (based on the alternating direction methods of multipliers) and numerical tests for examples from image deblurring. Thus it should be interesting for researchers from several disciplines. “What Are Higher-Order Networks” is a question raised and answered by Christian Bick, Elizabeth Gross, Heather A. Harrington, and Michael T. Schaub. In short, higher-order networks are a refurbishment of graphs, removing/overcoming some of the limitations of pairwise relationships by enabling the modeling of polyadic relations in real-world systems, such as reactions in biochemical systems with several species or reagents, or interactions of multiple people in social networks. The main topics of discussion are the understanding of the “shape” of data (by identifying and classifying topological and geometrical properties of the data), the modeling of relational data via higher-order networks, and network dynamical systems (describing couplings between dynamical units). The focus of the presentation is on the mathematical aspects of the topics, but a multitude of applications are mentioned. The impressive list of references comprises 316 entries. We believe the paper to be interesting for a broad audience.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2023-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71509712","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 65, Issue 3, Page 869-886, August 2023. Peixoto's structural stability and density theorems represent milestones in the modern theory of dynamical systems and their applications. Despite the importance of these theorems, they are often treated rather superficially, if at all, in upper level undergraduate courses on dynamical systems or differential equations. This is mainly because of the depth and length of the proofs. In this module, we formulate and prove the one-dimensional analogues of Peixoto's theorems in an intuitive and fairly simple way using only concepts and results that for the most part should be familiar to upper level undergraduate students in the mathematical sciences or related fields. The intention is to provide students who may be interested in further study in dynamical systems with an accessible one-dimensional treatment of structural stability theory that should help make Peixoto's theorems and their more recent generalizations easier to appreciate and understand. Further, we believe it is important and interesting for students to know the historical context of these discoveries since the mathematics was not done in isolation. The historical context is perhaps even more appropriate as it is the 100th anniversary of Marília Chaves Peixoto's and Maurício Matos Peixoto's births, February 24th and April 15th, 1921, respectively.
{"title":"The One-Dimensional Version of Peixoto's Structural Stability Theorem: A Calculus-Based Proof","authors":"Aminur Rahman, D. Blackmore","doi":"10.1137/21m1426572","DOIUrl":"https://doi.org/10.1137/21m1426572","url":null,"abstract":"SIAM Review, Volume 65, Issue 3, Page 869-886, August 2023. <br/> Peixoto's structural stability and density theorems represent milestones in the modern theory of dynamical systems and their applications. Despite the importance of these theorems, they are often treated rather superficially, if at all, in upper level undergraduate courses on dynamical systems or differential equations. This is mainly because of the depth and length of the proofs. In this module, we formulate and prove the one-dimensional analogues of Peixoto's theorems in an intuitive and fairly simple way using only concepts and results that for the most part should be familiar to upper level undergraduate students in the mathematical sciences or related fields. The intention is to provide students who may be interested in further study in dynamical systems with an accessible one-dimensional treatment of structural stability theory that should help make Peixoto's theorems and their more recent generalizations easier to appreciate and understand. Further, we believe it is important and interesting for students to know the historical context of these discoveries since the mathematics was not done in isolation. The historical context is perhaps even more appropriate as it is the 100th anniversary of Marília Chaves Peixoto's and Maurício Matos Peixoto's births, February 24th and April 15th, 1921, respectively.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2023-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71509659","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 65, Issue 3, Page 733-733, August 2023. The three articles in this issue's Research Spotlight section highlight the breadth of problems and approaches that have differential equations as a central component. In the first article, “Neural ODE Control for Classification, Approximation, and Transport,” authors Domènec Ruiz-Balet and Enrique Zuazua seek to expand understanding of some of the main properties of deep neural networks. To this end, the authors develop a dynamical control theoretical analysis of neural ordinary differential equations, a discretization of which is commonly known as a ResNet in machine learning. In this approach, time-dependent parameters are defined by piecewise-constant controls used to achieve targets associated with classification and regression tasks. A key aspect of the article's treatment is the reliance on an activation function characterization that only deforms one half space, leaving the other half space invariant; the rectified linear unit (ReLU) is a popular example of such an activation function. The authors derive constructive universal approximation results that can be used to understand how the complexity of the control depends on the target function's properties. Among other applications, these results are used to control a neural transport equation with the Wasserstein distance, common in optimal transport problems, measuring the approximation quality. Ruiz-Balet and Zuazua conclude with a number of open problems. Differential equation--based compartment models date back at least a century, when they were used to model the dynamics of malaria in a mixed population of humans and mosquitoes. Since then, compartment models have been used in areas far beyond epidemiology, typically with the simplifying assumption that each compartment is internally well mixed. As a consequence, all members in a compartment are treated the same, independent of how long they have resided in the compartment. In “Compartment Models with Memory,” authors Timothy Ginn and Lynn Schreyer expand the fields for which compartment models can provide insight by incorporating age in compartment in the underlying rate coefficients. This has the benefit of being able to account for a wide array of residence time distributions and comes at a cost of having to numerically solve a system of Volterra integral equations instead of a system of ordinary differential equations. The authors demonstrate and validate this approach on a number of examples and conclude by incorporating a delay in contagiousness of infected persons in a nonlinear SARS-CoV-2 transmission model. The authors also summarize several open questions based on this approach of allowing model parameters to be written as functions of age in compartment. “Does the Helmholtz Boundary Element Method Suffer from the Pollution Effect?” This is the question posed by (and the title of) the final Research Spotlights article in this issue. Authors Jeffrey Galkowski
{"title":"Research Spotlights","authors":"Stefan M. Wild","doi":"10.1137/23n975739","DOIUrl":"https://doi.org/10.1137/23n975739","url":null,"abstract":"SIAM Review, Volume 65, Issue 3, Page 733-733, August 2023. <br/> The three articles in this issue's Research Spotlight section highlight the breadth of problems and approaches that have differential equations as a central component. In the first article, “Neural ODE Control for Classification, Approximation, and Transport,” authors Domènec Ruiz-Balet and Enrique Zuazua seek to expand understanding of some of the main properties of deep neural networks. To this end, the authors develop a dynamical control theoretical analysis of neural ordinary differential equations, a discretization of which is commonly known as a ResNet in machine learning. In this approach, time-dependent parameters are defined by piecewise-constant controls used to achieve targets associated with classification and regression tasks. A key aspect of the article's treatment is the reliance on an activation function characterization that only deforms one half space, leaving the other half space invariant; the rectified linear unit (ReLU) is a popular example of such an activation function. The authors derive constructive universal approximation results that can be used to understand how the complexity of the control depends on the target function's properties. Among other applications, these results are used to control a neural transport equation with the Wasserstein distance, common in optimal transport problems, measuring the approximation quality. Ruiz-Balet and Zuazua conclude with a number of open problems. Differential equation--based compartment models date back at least a century, when they were used to model the dynamics of malaria in a mixed population of humans and mosquitoes. Since then, compartment models have been used in areas far beyond epidemiology, typically with the simplifying assumption that each compartment is internally well mixed. As a consequence, all members in a compartment are treated the same, independent of how long they have resided in the compartment. In “Compartment Models with Memory,” authors Timothy Ginn and Lynn Schreyer expand the fields for which compartment models can provide insight by incorporating age in compartment in the underlying rate coefficients. This has the benefit of being able to account for a wide array of residence time distributions and comes at a cost of having to numerically solve a system of Volterra integral equations instead of a system of ordinary differential equations. The authors demonstrate and validate this approach on a number of examples and conclude by incorporating a delay in contagiousness of infected persons in a nonlinear SARS-CoV-2 transmission model. The authors also summarize several open questions based on this approach of allowing model parameters to be written as functions of age in compartment. “Does the Helmholtz Boundary Element Method Suffer from the Pollution Effect?” This is the question posed by (and the title of) the final Research Spotlights article in this issue. Authors Jeffrey Galkowski ","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2023-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71509716","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 65, Issue 3, Page 829-829, August 2023. The SIGEST article in this issue, which comes from the SIAM/ASA Journal on Uncertainty Quantification, is “Bayesian Inverse Problems Are Usually Well-Posed,” by Jonas Latz. The author investigates the well-posedness of Bayesian approaches to inverse problems, generalizing the framework of well-posedness introduced by Andrew Stuart to a set of weaker assumptions. Well-posedness here is understood in the sense of Hadamard, that is, a solution exists, is unique, and continuously depends on the input data. Inverse problems are typically ill-posed due to properties of the model, a lack of data, and measurement noise. The Bayesian approach to inverse problems reformulates the quest for a solution to the inverse problem in terms of a quest for its posterior distribution, which is determined by the data likelihood and prior distribution of the solution, and which in contrast to the inverse problem itself should be well-posed. In the Bayesian context, well-posedness typically relates to existence, uniqueness, and Lipschitz continuity of the posterior distribution with respect to the data in the so-called Hellinger distance. In many practical applications such well-posedness is difficult, if not impossible, to verify. Moreover, the choice of the Hellinger distance as the right metric might not always be the best fitted depending on the problem at hand. This sets the starting point for the paper where the author introduces a new framework for well-posedness of Bayesian inverse problems in which he shows existence, uniqueness, and continuity with respect to various metrics for a large class of Bayesian inverse problems, with conditions that are either nonrestrictive or verifiable in practical settings. This paper gives a strong new mathematical foundation for Bayesian inverse problems. The underlying statistical and probabilistic concepts are explained comprehensively and comprehensibly and, thus, in a way that opens up the Bayesian approach for a large readership. For the SIGEST version of the paper the author introduced more background material to make it more accessible to a general audience and extended the conclusion and outlook section, summarizing developments in the field that happened since the publication of the original work and discussing future research directions.
{"title":"SIGEST","authors":"The Editors","doi":"10.1137/23n975740","DOIUrl":"https://doi.org/10.1137/23n975740","url":null,"abstract":"SIAM Review, Volume 65, Issue 3, Page 829-829, August 2023. <br/> The SIGEST article in this issue, which comes from the SIAM/ASA Journal on Uncertainty Quantification, is “Bayesian Inverse Problems Are Usually Well-Posed,” by Jonas Latz. The author investigates the well-posedness of Bayesian approaches to inverse problems, generalizing the framework of well-posedness introduced by Andrew Stuart to a set of weaker assumptions. Well-posedness here is understood in the sense of Hadamard, that is, a solution exists, is unique, and continuously depends on the input data. Inverse problems are typically ill-posed due to properties of the model, a lack of data, and measurement noise. The Bayesian approach to inverse problems reformulates the quest for a solution to the inverse problem in terms of a quest for its posterior distribution, which is determined by the data likelihood and prior distribution of the solution, and which in contrast to the inverse problem itself should be well-posed. In the Bayesian context, well-posedness typically relates to existence, uniqueness, and Lipschitz continuity of the posterior distribution with respect to the data in the so-called Hellinger distance. In many practical applications such well-posedness is difficult, if not impossible, to verify. Moreover, the choice of the Hellinger distance as the right metric might not always be the best fitted depending on the problem at hand. This sets the starting point for the paper where the author introduces a new framework for well-posedness of Bayesian inverse problems in which he shows existence, uniqueness, and continuity with respect to various metrics for a large class of Bayesian inverse problems, with conditions that are either nonrestrictive or verifiable in practical settings. This paper gives a strong new mathematical foundation for Bayesian inverse problems. The underlying statistical and probabilistic concepts are explained comprehensively and comprehensibly and, thus, in a way that opens up the Bayesian approach for a large readership. For the SIGEST version of the paper the author introduced more background material to make it more accessible to a general audience and extended the conclusion and outlook section, summarizing developments in the field that happened since the publication of the original work and discussing future research directions.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2023-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71509719","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Review, Volume 65, Issue 3, Page 601-685, August 2023. Over the last 30 years a plethora of variational regularization models for image reconstruction have been proposed and thoroughly inspected by the applied mathematics community. Among them, the pioneering prototype often taught and learned in basic courses in mathematical image processing is the celebrated Rudin--Osher--Fatemi (ROF) model [L. I. Rudin, S. Osher, and E. Fatemi, Phys. D, 60 (1992), pp. 259--268], which relies on the minimization of the edge-preserving total variation (TV) seminorm as a regularization term. Despite its (often limiting) simplicity, this model is still very much employed in many applications and used as a benchmark for assessing the performance of modern learning-based image reconstruction approaches, thanks to its thorough analytical and numerical understanding. Among the many extensions to TV proposed over the years, a large class is based on the concept of space variance. Space-variant models can indeed overcome the intrinsic inability of TV to describe local features (strength, sharpness, directionality) by means of an adaptive mathematical modeling which accommodates local regularization weighting, variable smoothness, and anisotropy. Those ideas can further be cast in the flexible Bayesian framework of generalized Gaussian distributions and combined with maximum likelihood and hierarchical optimization approaches for efficient hyperparameter estimation. In this work, we review and connect the major contributions in the field of space-variant TV-type image reconstruction models, focusing, in particular, on their Bayesian interpretation which paves the way to new exciting and unexplored research directions.
{"title":"On and Beyond Total Variation Regularization in Imaging: The Role of Space Variance","authors":"Monica Pragliola, Luca Calatroni, Alessandro Lanza, Fiorella Sgallari","doi":"10.1137/21m1410683","DOIUrl":"https://doi.org/10.1137/21m1410683","url":null,"abstract":"SIAM Review, Volume 65, Issue 3, Page 601-685, August 2023. <br/> Over the last 30 years a plethora of variational regularization models for image reconstruction have been proposed and thoroughly inspected by the applied mathematics community. Among them, the pioneering prototype often taught and learned in basic courses in mathematical image processing is the celebrated Rudin--Osher--Fatemi (ROF) model [L. I. Rudin, S. Osher, and E. Fatemi, Phys. D, 60 (1992), pp. 259--268], which relies on the minimization of the edge-preserving total variation (TV) seminorm as a regularization term. Despite its (often limiting) simplicity, this model is still very much employed in many applications and used as a benchmark for assessing the performance of modern learning-based image reconstruction approaches, thanks to its thorough analytical and numerical understanding. Among the many extensions to TV proposed over the years, a large class is based on the concept of space variance. Space-variant models can indeed overcome the intrinsic inability of TV to describe local features (strength, sharpness, directionality) by means of an adaptive mathematical modeling which accommodates local regularization weighting, variable smoothness, and anisotropy. Those ideas can further be cast in the flexible Bayesian framework of generalized Gaussian distributions and combined with maximum likelihood and hierarchical optimization approaches for efficient hyperparameter estimation. In this work, we review and connect the major contributions in the field of space-variant TV-type image reconstruction models, focusing, in particular, on their Bayesian interpretation which paves the way to new exciting and unexplored research directions.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2023-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71509713","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Christian Bick, Elizabeth Gross, Heather A. Harrington, Michael T. Schaub
SIAM Review, Volume 65, Issue 3, Page 686-731, August 2023. Network-based modeling of complex systems and data using the language of graphs has become an essential topic across a range of different disciplines. Arguably, this graph-based perspective derives its success from the relative simplicity of graphs: A graph consists of nothing more than a set of vertices and a set of edges, describing relationships between pairs of such vertices. This simple combinatorial structure makes graphs interpretable and flexible modeling tools. The simplicity of graphs as system models, however, has been scrutinized in the literature recently. Specifically, it has been argued from a variety of different angles that there is a need for higher-order networks, which go beyond the paradigm of modeling pairwise relationships, as encapsulated by graphs. In this survey article we take stock of these recent developments. Our goals are to clarify (i) what higher-order networks are, (ii) why these are interesting objects of study, and (iii) how they can be used in applications.
{"title":"What Are Higher-Order Networks?","authors":"Christian Bick, Elizabeth Gross, Heather A. Harrington, Michael T. Schaub","doi":"10.1137/21m1414024","DOIUrl":"https://doi.org/10.1137/21m1414024","url":null,"abstract":"SIAM Review, Volume 65, Issue 3, Page 686-731, August 2023. <br/> Network-based modeling of complex systems and data using the language of graphs has become an essential topic across a range of different disciplines. Arguably, this graph-based perspective derives its success from the relative simplicity of graphs: A graph consists of nothing more than a set of vertices and a set of edges, describing relationships between pairs of such vertices. This simple combinatorial structure makes graphs interpretable and flexible modeling tools. The simplicity of graphs as system models, however, has been scrutinized in the literature recently. Specifically, it has been argued from a variety of different angles that there is a need for higher-order networks, which go beyond the paradigm of modeling pairwise relationships, as encapsulated by graphs. In this survey article we take stock of these recent developments. Our goals are to clarify (i) what higher-order networks are, (ii) why these are interesting objects of study, and (iii) how they can be used in applications.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":null,"pages":null},"PeriodicalIF":10.2,"publicationDate":"2023-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71509714","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号