Aksel K. Rasmussen, Fanny Seizilles, Mark Girolami, Ieva Kazlauskaite
SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 1050-1084, September 2024. Abstract.In this paper, we investigate the Bayesian approach to inverse Robin problems. These are problems for certain elliptic boundary value problems of determining a Robin coefficient on a hidden part of the boundary from Cauchy data on the observable part. Such a nonlinear inverse problem arises naturally in the initialization of large-scale ice sheet models that are crucial in climate and sea-level predictions. We motivate the Bayesian approach for a prototypical Robin inverse problem by showing that the posterior mean converges in probability to the data-generating ground truth as the number of observations increases. Related to the stability theory for inverse Robin problems, we establish a logarithmic convergence rate for Sobolev-regular Robin coefficients, whereas for analytic coefficients we can attain an algebraic rate. The use of rescaled analytic Gaussian priors in posterior consistency for nonlinear inverse problems is new and may be of separate interest in other inverse problems. Our numerical results illustrate the convergence property in two observation settings.
{"title":"The Bayesian Approach to Inverse Robin Problems","authors":"Aksel K. Rasmussen, Fanny Seizilles, Mark Girolami, Ieva Kazlauskaite","doi":"10.1137/23m1620624","DOIUrl":"https://doi.org/10.1137/23m1620624","url":null,"abstract":"SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 1050-1084, September 2024. <br/> Abstract.In this paper, we investigate the Bayesian approach to inverse Robin problems. These are problems for certain elliptic boundary value problems of determining a Robin coefficient on a hidden part of the boundary from Cauchy data on the observable part. Such a nonlinear inverse problem arises naturally in the initialization of large-scale ice sheet models that are crucial in climate and sea-level predictions. We motivate the Bayesian approach for a prototypical Robin inverse problem by showing that the posterior mean converges in probability to the data-generating ground truth as the number of observations increases. Related to the stability theory for inverse Robin problems, we establish a logarithmic convergence rate for Sobolev-regular Robin coefficients, whereas for analytic coefficients we can attain an algebraic rate. The use of rescaled analytic Gaussian priors in posterior consistency for nonlinear inverse problems is new and may be of separate interest in other inverse problems. Our numerical results illustrate the convergence property in two observation settings.","PeriodicalId":56064,"journal":{"name":"Siam-Asa Journal on Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":2.0,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142226812","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Thomas O. Dixon, James E. Warner, Geoffrey F. Bomarito, Alex A. Gorodetsky
SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 1005-1049, September 2024. Abstract.We provide a collection of results on covariance expressions between Monte Carlo–based multioutput mean, variance, and Sobol main effect variance estimators from an ensemble of models. These covariances can be used within multifidelity uncertainty quantification strategies that seek to reduce the estimator variance of high-fidelity Monte Carlo estimators with an ensemble of low-fidelity models. Such covariance expressions are required within approaches such as the approximate control variate and multilevel best linear unbiased estimator. While the literature provides these expressions for some single-output cases such as mean and variance, our results are relevant to both multiple function outputs and multiple statistics across any sampling strategy. Following the description of these results, we use them within an approximate control variate scheme to show that leveraging multiple outputs can dramatically reduce estimator variance compared to single-output approaches. Synthetic examples are used to highlight the effects of optimal sample allocation and pilot sample estimation. A flight-trajectory simulation of entry, descent, and landing is used to demonstrate multioutput estimation in practical applications.
{"title":"Covariance Expressions for Multifidelity Sampling with Multioutput, Multistatistic Estimators: Application to Approximate Control Variates","authors":"Thomas O. Dixon, James E. Warner, Geoffrey F. Bomarito, Alex A. Gorodetsky","doi":"10.1137/23m1607994","DOIUrl":"https://doi.org/10.1137/23m1607994","url":null,"abstract":"SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 1005-1049, September 2024. <br/> Abstract.We provide a collection of results on covariance expressions between Monte Carlo–based multioutput mean, variance, and Sobol main effect variance estimators from an ensemble of models. These covariances can be used within multifidelity uncertainty quantification strategies that seek to reduce the estimator variance of high-fidelity Monte Carlo estimators with an ensemble of low-fidelity models. Such covariance expressions are required within approaches such as the approximate control variate and multilevel best linear unbiased estimator. While the literature provides these expressions for some single-output cases such as mean and variance, our results are relevant to both multiple function outputs and multiple statistics across any sampling strategy. Following the description of these results, we use them within an approximate control variate scheme to show that leveraging multiple outputs can dramatically reduce estimator variance compared to single-output approaches. Synthetic examples are used to highlight the effects of optimal sample allocation and pilot sample estimation. A flight-trajectory simulation of entry, descent, and landing is used to demonstrate multioutput estimation in practical applications.","PeriodicalId":56064,"journal":{"name":"Siam-Asa Journal on Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":2.0,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142213297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 964-1004, September 2024. Abstract.Partial differential equations (PDEs) are widely used for the description of physical and engineering phenomena. Some key parameters involved in PDEs, which represent certain physical properties with important scientific interpretations, are difficult or even impossible to measure directly. Estimating these parameters from noisy and sparse experimental data of related physical quantities is an important task. Many methods for PDE parameter inference involve a large number of evaluations for numerical solutions to PDEs through algorithms such as the finite element method, which can be time consuming, especially for nonlinear PDEs. In this paper, we propose a novel method for the inference of unknown parameters in PDEs, called the PDE-informed Gaussian process (PIGP)–based parameter inference method. Through modeling the PDE solution as a Gaussian process (GP), we derive the manifold constraints induced by the (linear) PDE structure such that, under the constraints, the GP satisfies the PDE. For nonlinear PDEs, we propose an augmentation method that transforms the nonlinear PDE into an equivalent PDE system linear in all derivatives, which our PIGP-based method can handle. The proposed method can be applied to a broad spectrum of nonlinear PDEs. The PIGP-based method can be applied to multidimensional PDE systems and PDE systems with unobserved components. Like conventional Bayesian approaches, the method can provide uncertainty quantification for both the unknown parameters and the PDE solution. The PIGP-based method also completely bypasses the numerical solver for PDEs. The proposed method is demonstrated through several application examples from different areas.
{"title":"Parameter Inference Based on Gaussian Processes Informed by Nonlinear Partial Differential Equations","authors":"Zhaohui Li, Shihao Yang, C. F. Jeff Wu","doi":"10.1137/22m1514131","DOIUrl":"https://doi.org/10.1137/22m1514131","url":null,"abstract":"SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 964-1004, September 2024. <br/> Abstract.Partial differential equations (PDEs) are widely used for the description of physical and engineering phenomena. Some key parameters involved in PDEs, which represent certain physical properties with important scientific interpretations, are difficult or even impossible to measure directly. Estimating these parameters from noisy and sparse experimental data of related physical quantities is an important task. Many methods for PDE parameter inference involve a large number of evaluations for numerical solutions to PDEs through algorithms such as the finite element method, which can be time consuming, especially for nonlinear PDEs. In this paper, we propose a novel method for the inference of unknown parameters in PDEs, called the PDE-informed Gaussian process (PIGP)–based parameter inference method. Through modeling the PDE solution as a Gaussian process (GP), we derive the manifold constraints induced by the (linear) PDE structure such that, under the constraints, the GP satisfies the PDE. For nonlinear PDEs, we propose an augmentation method that transforms the nonlinear PDE into an equivalent PDE system linear in all derivatives, which our PIGP-based method can handle. The proposed method can be applied to a broad spectrum of nonlinear PDEs. The PIGP-based method can be applied to multidimensional PDE systems and PDE systems with unobserved components. Like conventional Bayesian approaches, the method can provide uncertainty quantification for both the unknown parameters and the PDE solution. The PIGP-based method also completely bypasses the numerical solver for PDEs. The proposed method is demonstrated through several application examples from different areas.","PeriodicalId":56064,"journal":{"name":"Siam-Asa Journal on Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":2.0,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142213379","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
D. Elfverson, R. Scheichl, S. Weissmann, F. A. Diaz De La O
SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 932-963, September 2024. Abstract. In this work we propose an adaptive multilevel version of subset simulation to estimate the probability of rare events for complex physical systems. Given a sequence of nested failure domains of increasing size, the rare event probability is expressed as a product of conditional probabilities. The proposed new estimator uses different model resolutions and varying numbers of samples across the hierarchy of nested failure sets. In order to dramatically reduce the computational cost, we construct the intermediate failure sets such that only a small number of expensive high-resolution model evaluations are needed, whilst the majority of samples can be taken from inexpensive low-resolution simulations. A key idea in our new estimator is the use of a posteriori error estimators combined with a selective mesh refinement strategy to guarantee the critical subset property that may be violated when changing model resolution from one failure set to the next. The efficiency gains and the statistical properties of the estimator are investigated both theoretically via shaking transformations, as well as numerically. On a model problem from subsurface flow, the new multilevel estimator achieves gains of more than a factor 60 over standard subset simulation for a practically relevant relative error of 25%.
{"title":"Adaptive Multilevel Subset Simulation with Selective Refinement","authors":"D. Elfverson, R. Scheichl, S. Weissmann, F. A. Diaz De La O","doi":"10.1137/22m1515240","DOIUrl":"https://doi.org/10.1137/22m1515240","url":null,"abstract":"SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 932-963, September 2024. <br/> Abstract. In this work we propose an adaptive multilevel version of subset simulation to estimate the probability of rare events for complex physical systems. Given a sequence of nested failure domains of increasing size, the rare event probability is expressed as a product of conditional probabilities. The proposed new estimator uses different model resolutions and varying numbers of samples across the hierarchy of nested failure sets. In order to dramatically reduce the computational cost, we construct the intermediate failure sets such that only a small number of expensive high-resolution model evaluations are needed, whilst the majority of samples can be taken from inexpensive low-resolution simulations. A key idea in our new estimator is the use of a posteriori error estimators combined with a selective mesh refinement strategy to guarantee the critical subset property that may be violated when changing model resolution from one failure set to the next. The efficiency gains and the statistical properties of the estimator are investigated both theoretically via shaking transformations, as well as numerically. On a model problem from subsurface flow, the new multilevel estimator achieves gains of more than a factor 60 over standard subset simulation for a practically relevant relative error of 25%.","PeriodicalId":56064,"journal":{"name":"Siam-Asa Journal on Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":2.0,"publicationDate":"2024-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142213380","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Niklas Baumgarten, Sebastian Krumscheid, Christian Wieners
SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 901-931, September 2024. Abstract.We present a novel variant of the multilevel Monte Carlo method that effectively utilizes a reserved computational budget on a high-performance computing system to minimize the mean squared error. Our approach combines concepts of the continuation multilevel Monte Carlo method with dynamic programming techniques following Bellman’s optimality principle and a new parallelization strategy based on a single distributed data structure. Additionally, we establish a theoretical bound on the error reduction on a parallel computing cluster and provide empirical evidence that the proposed method adheres to this bound. We implement, test, and benchmark the approach on computationally demanding problems, focusing on its application to acoustic wave propagation in high-dimensional random media.
{"title":"A Fully Parallelized and Budgeted Multilevel Monte Carlo Method and the Application to Acoustic Waves","authors":"Niklas Baumgarten, Sebastian Krumscheid, Christian Wieners","doi":"10.1137/23m1588354","DOIUrl":"https://doi.org/10.1137/23m1588354","url":null,"abstract":"SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 901-931, September 2024. <br/> Abstract.We present a novel variant of the multilevel Monte Carlo method that effectively utilizes a reserved computational budget on a high-performance computing system to minimize the mean squared error. Our approach combines concepts of the continuation multilevel Monte Carlo method with dynamic programming techniques following Bellman’s optimality principle and a new parallelization strategy based on a single distributed data structure. Additionally, we establish a theoretical bound on the error reduction on a parallel computing cluster and provide empirical evidence that the proposed method adheres to this bound. We implement, test, and benchmark the approach on computationally demanding problems, focusing on its application to acoustic wave propagation in high-dimensional random media.","PeriodicalId":56064,"journal":{"name":"Siam-Asa Journal on Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":2.0,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142213382","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ricardo Baptista, Bamdad Hosseini, Nikola B. Kovachki, Youssef M. Marzouk
SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 868-900, September 2024. Abstract.We present a novel framework for conditional sampling of probability measures, using block triangular transport maps. We develop the theoretical foundations of block triangular transport in a Banach space setting, establishing general conditions under which conditional sampling can be achieved and drawing connections between monotone block triangular maps and optimal transport. Based on this theory, we then introduce a computational approach, called monotone generative adversarial networks (M-GANs), to learn suitable block triangular maps. Our algorithm uses only samples from the underlying joint probability measure and is hence likelihood-free. Numerical experiments with M-GAN demonstrate accurate sampling of conditional measures in synthetic examples, Bayesian inverse problems involving ordinary and partial differential equations, and probabilistic image inpainting.
{"title":"Conditional Sampling with Monotone GANs: From Generative Models to Likelihood-Free Inference","authors":"Ricardo Baptista, Bamdad Hosseini, Nikola B. Kovachki, Youssef M. Marzouk","doi":"10.1137/23m1581546","DOIUrl":"https://doi.org/10.1137/23m1581546","url":null,"abstract":"SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 868-900, September 2024. <br/> Abstract.We present a novel framework for conditional sampling of probability measures, using block triangular transport maps. We develop the theoretical foundations of block triangular transport in a Banach space setting, establishing general conditions under which conditional sampling can be achieved and drawing connections between monotone block triangular maps and optimal transport. Based on this theory, we then introduce a computational approach, called monotone generative adversarial networks (M-GANs), to learn suitable block triangular maps. Our algorithm uses only samples from the underlying joint probability measure and is hence likelihood-free. Numerical experiments with M-GAN demonstrate accurate sampling of conditional measures in synthetic examples, Bayesian inverse problems involving ordinary and partial differential equations, and probabilistic image inpainting.","PeriodicalId":56064,"journal":{"name":"Siam-Asa Journal on Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":2.0,"publicationDate":"2024-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141934373","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 842-867, September 2024. Abstract.Harmonizable processes can be represented by sums of harmonics with random coefficients, which are correlated rather than uncorrelated as for weakly stationary processes. Harmonizable processes are characterized in the second moment sense by their generalized spectral density functions. It is shown that harmonizable processes admit spectral representations and can be band limited and/or narrow band; samples of harmonizable Gaussian processes can be generated by algorithms similar to those used to generate samples of stationary Gaussian processes; accurate finite dimensional (FD) surrogates, i.e., deterministic functions of time and finite sets of random variables, can be constructed for harmonizable processes; and, under mild conditions, a broad range of nonstationary processes are harmonizable. Numerical illustrations, including various nonstationary processes and outputs of linear systems to random inputs, are presented to demonstrate the versatility of harmonizable processes.
{"title":"Harmonizable Nonstationary Processes","authors":"Mircea Grigoriu","doi":"10.1137/22m1544580","DOIUrl":"https://doi.org/10.1137/22m1544580","url":null,"abstract":"SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 842-867, September 2024. <br/> Abstract.Harmonizable processes can be represented by sums of harmonics with random coefficients, which are correlated rather than uncorrelated as for weakly stationary processes. Harmonizable processes are characterized in the second moment sense by their generalized spectral density functions. It is shown that harmonizable processes admit spectral representations and can be band limited and/or narrow band; samples of harmonizable Gaussian processes can be generated by algorithms similar to those used to generate samples of stationary Gaussian processes; accurate finite dimensional (FD) surrogates, i.e., deterministic functions of time and finite sets of random variables, can be constructed for harmonizable processes; and, under mild conditions, a broad range of nonstationary processes are harmonizable. Numerical illustrations, including various nonstationary processes and outputs of linear systems to random inputs, are presented to demonstrate the versatility of harmonizable processes.","PeriodicalId":56064,"journal":{"name":"Siam-Asa Journal on Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":2.0,"publicationDate":"2024-07-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141863452","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 812-841, September 2024. Abstract.In this paper, we study the Boltzmann equation with uncertainties and prove that the spectral convergence of the semi-discretized numerical system holds in a combined velocity and random space, where the Fourier spectral method is applied for approximation in the velocity space, whereas the generalized polynomial chaos (gPC)-based stochastic Galerkin (SG) method is employed to discretize the random variable. Our proof is based on a delicate energy estimate for showing the well-posedness of the numerical solution as well as a rigorous control of its negative part in our well-designed functional space that involves high-order derivatives of both the velocity and random variables. This paper rigorously justifies the statement proposed in Remark 4.4 of [J. Hu and S. Jin, J. Comput. Phys., 315 (2016), pp. 150–168].
SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 812-841, September 2024. 摘要.本文研究了具有不确定性的玻尔兹曼方程，并证明了半离散化数值系统的谱收敛性在速度空间和随机空间的组合中成立，其中傅立叶谱方法用于速度空间的逼近，而基于广义多项式混沌（gPC）的随机伽勒金（SG）方法用于随机变量的离散化。我们的证明基于一个微妙的能量估算，以显示数值解的好求解性，以及在我们精心设计的函数空间中对其负部分的严格控制，该函数空间涉及速度和随机变量的高阶导数。本文严格证明了[J. Hu and S. Jin, J. Comput. Phys., 315 (2016), pp.］
{"title":"Spectral Convergence of a Semi-discretized Numerical System for the Spatially Homogeneous Boltzmann Equation with Uncertainties","authors":"Liu Liu, Kunlun Qi","doi":"10.1137/24m1638483","DOIUrl":"https://doi.org/10.1137/24m1638483","url":null,"abstract":"SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 812-841, September 2024. <br/> Abstract.In this paper, we study the Boltzmann equation with uncertainties and prove that the spectral convergence of the semi-discretized numerical system holds in a combined velocity and random space, where the Fourier spectral method is applied for approximation in the velocity space, whereas the generalized polynomial chaos (gPC)-based stochastic Galerkin (SG) method is employed to discretize the random variable. Our proof is based on a delicate energy estimate for showing the well-posedness of the numerical solution as well as a rigorous control of its negative part in our well-designed functional space that involves high-order derivatives of both the velocity and random variables. This paper rigorously justifies the statement proposed in Remark 4.4 of [J. Hu and S. Jin, J. Comput. Phys., 315 (2016), pp. 150–168].","PeriodicalId":56064,"journal":{"name":"Siam-Asa Journal on Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":2.0,"publicationDate":"2024-07-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141781343","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hossein Mohammadi, Peter Challenor, Marc Goodfellow
SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 788-811, September 2024. Abstract.A Gaussian process (GP)-based methodology is proposed to emulate complex dynamical computer models (or simulators). The method relies on emulating the numerical flow map of the system over an initial (short) time step, where the flow map is a function that describes the evolution of the system from an initial condition to a subsequent value at the next time step. This yields a probabilistic distribution over the entire flow map function, with each draw offering an approximation to the flow map. The model output time series is then predicted (under the Markov assumption) by drawing a sample from the emulated flow map (i.e., its posterior distribution) and using it to iterate from the initial condition ahead in time. Repeating this procedure with multiple such draws creates a distribution over the time series. The mean and variance of this distribution at a specific time point serve as the model output prediction and the associated uncertainty, respectively. However, drawing a GP posterior sample that represents the underlying function across its entire domain is computationally infeasible, given the infinite-dimensional nature of this object. To overcome this limitation, one can generate such a sample in an approximate manner using random Fourier features (RFF). RFF is an efficient technique for approximating the kernel and generating GP samples, offering both computational efficiency and theoretical guarantees. The proposed method is applied to emulate several dynamic nonlinear simulators including the well-known Lorenz and van der Pol models. The results suggest that our approach has a promising predictive performance and the associated uncertainty can capture the dynamics of the system appropriately.
SIAM/ASA 不确定性量化期刊》，第 12 卷，第 3 期，第 788-811 页，2024 年 9 月。 摘要：本文提出了一种基于高斯过程（GP）的方法来模拟复杂的动态计算机模型（或模拟器）。该方法依赖于模拟系统在初始（短）时间步上的数值流图，其中流图是描述系统从初始条件到下一时间步的后续值的演变的函数。这就产生了整个流图函数的概率分布，每次绘制都提供了流图的近似值。然后，根据马尔可夫假设，从仿真流图（即其后验分布）中抽取样本，并利用它从初始条件向前迭代，从而预测模型输出时间序列（根据马尔可夫假设）。多次重复这一过程，就会在时间序列上形成一个分布。该分布在特定时间点的均值和方差分别作为模型输出预测值和相关的不确定性。然而，考虑到 GP 后验样本的无限维性质，要绘制一个能代表整个领域的基础函数的 GP 后验样本在计算上是不可行的。为了克服这一限制，我们可以使用随机傅立叶特征（RFF）近似生成这样的样本。随机傅里叶特征是一种近似内核和生成 GP 样本的高效技术，既能提高计算效率，又能提供理论保证。所提出的方法被应用于模拟多个动态非线性模拟器，包括著名的 Lorenz 和 van der Pol 模型。结果表明，我们的方法具有良好的预测性能，相关的不确定性可以适当捕捉系统的动态。
{"title":"Emulating Complex Dynamical Simulators with Random Fourier Features","authors":"Hossein Mohammadi, Peter Challenor, Marc Goodfellow","doi":"10.1137/22m147339x","DOIUrl":"https://doi.org/10.1137/22m147339x","url":null,"abstract":"SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 788-811, September 2024. <br/> Abstract.A Gaussian process (GP)-based methodology is proposed to emulate complex dynamical computer models (or simulators). The method relies on emulating the numerical flow map of the system over an initial (short) time step, where the flow map is a function that describes the evolution of the system from an initial condition to a subsequent value at the next time step. This yields a probabilistic distribution over the entire flow map function, with each draw offering an approximation to the flow map. The model output time series is then predicted (under the Markov assumption) by drawing a sample from the emulated flow map (i.e., its posterior distribution) and using it to iterate from the initial condition ahead in time. Repeating this procedure with multiple such draws creates a distribution over the time series. The mean and variance of this distribution at a specific time point serve as the model output prediction and the associated uncertainty, respectively. However, drawing a GP posterior sample that represents the underlying function across its entire domain is computationally infeasible, given the infinite-dimensional nature of this object. To overcome this limitation, one can generate such a sample in an approximate manner using random Fourier features (RFF). RFF is an efficient technique for approximating the kernel and generating GP samples, offering both computational efficiency and theoretical guarantees. The proposed method is applied to emulate several dynamic nonlinear simulators including the well-known Lorenz and van der Pol models. The results suggest that our approach has a promising predictive performance and the associated uncertainty can capture the dynamics of the system appropriately.","PeriodicalId":56064,"journal":{"name":"Siam-Asa Journal on Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":2.0,"publicationDate":"2024-07-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141739021","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 759-787, September 2024. Abstract.Sparse Bayesian learning (SBL) models are extensively used in signal processing and machine learning for promoting sparsity through hierarchical priors. The hyperparameters in SBL models are crucial for the model’s performance, but they are often difficult to estimate due to the nonconvexity and the high-dimensionality of the associated objective function. This paper presents a comprehensive framework for hyperparameter estimation in SBL models, encompassing well-known algorithms such as the expectation-maximization, MacKay, and convex bounding algorithms. These algorithms are cohesively interpreted within an alternating minimization and linearization (AML) paradigm, distinguished by their unique linearized surrogate functions. Additionally, a novel algorithm within the AML framework is introduced, showing enhanced efficiency, especially under low signal noise ratios. This is further improved by a new alternating minimization and quadratic approximation paradigm, which includes a proximal regularization term. The paper substantiates these advancements with thorough convergence analysis and numerical experiments, demonstrating the algorithm’s effectiveness in various noise conditions and signal-to-noise ratios.
{"title":"Hyperparameter Estimation for Sparse Bayesian Learning Models","authors":"Feng Yu, Lixin Shen, Guohui Song","doi":"10.1137/24m162844x","DOIUrl":"https://doi.org/10.1137/24m162844x","url":null,"abstract":"SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 759-787, September 2024. <br/> Abstract.Sparse Bayesian learning (SBL) models are extensively used in signal processing and machine learning for promoting sparsity through hierarchical priors. The hyperparameters in SBL models are crucial for the model’s performance, but they are often difficult to estimate due to the nonconvexity and the high-dimensionality of the associated objective function. This paper presents a comprehensive framework for hyperparameter estimation in SBL models, encompassing well-known algorithms such as the expectation-maximization, MacKay, and convex bounding algorithms. These algorithms are cohesively interpreted within an alternating minimization and linearization (AML) paradigm, distinguished by their unique linearized surrogate functions. Additionally, a novel algorithm within the AML framework is introduced, showing enhanced efficiency, especially under low signal noise ratios. This is further improved by a new alternating minimization and quadratic approximation paradigm, which includes a proximal regularization term. The paper substantiates these advancements with thorough convergence analysis and numerical experiments, demonstrating the algorithm’s effectiveness in various noise conditions and signal-to-noise ratios.","PeriodicalId":56064,"journal":{"name":"Siam-Asa Journal on Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":2.0,"publicationDate":"2024-07-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141739022","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号