SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page A2848-A2873, October 2024. Abstract. We propose a practical finite volume method on cut cells using state redistribution. Our algorithm is provably monotone, total variation diminishing, and GKS (Gustafsson, Kreiss, Sundström) stable in many situations, and shuts off continuously as the cut cell size approaches a target value. Our analysis reveals why original state redistribution works so well: it results in a monotone scheme for most configurations, though at times subject to a slightly smaller CFL condition. Our analysis also explains why a premerging step is beneficial. We show computational experiments in two and three dimensions.
{"title":"A New Provably Stable Weighted State Redistribution Algorithm","authors":"Marsha Berger, Andrew Giuliani","doi":"10.1137/23m1597484","DOIUrl":"https://doi.org/10.1137/23m1597484","url":null,"abstract":"SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page A2848-A2873, October 2024. <br/> Abstract. We propose a practical finite volume method on cut cells using state redistribution. Our algorithm is provably monotone, total variation diminishing, and GKS (Gustafsson, Kreiss, Sundström) stable in many situations, and shuts off continuously as the cut cell size approaches a target value. Our analysis reveals why original state redistribution works so well: it results in a monotone scheme for most configurations, though at times subject to a slightly smaller CFL condition. Our analysis also explains why a premerging step is beneficial. We show computational experiments in two and three dimensions.","PeriodicalId":49526,"journal":{"name":"SIAM Journal on Scientific Computing","volume":"36 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142217593","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page C557-C582, October 2024. Abstract. Deep neural networks (DNNs) and, in particular, convolutional neural networks (CNNs) have brought significant advances in a wide range of modern computer application problems. However, the increasing availability of large numbers of datasets and the increasing available computational power of modern computers have led to steady growth in the complexity and size of DNN and CNN models, respectively, and thus, to longer training times. Hence, various methods and attempts have been developed to accelerate and parallelize the training of complex network architectures. In this work, a novel CNN-DNN architecture is proposed that naturally supports a model parallel training strategy and that is loosely inspired by two-level domain decomposition methods (DDMs). First, local CNN models, that is, subnetworks, are defined that operate on overlapping or nonoverlapping parts of the input data, for example, subimages. The subnetworks can be trained completely in parallel and independently of each other. Each subnetwork then outputs a local decision for the given machine learning problem which is exclusively based on the respective local input data. Subsequently, in a second step, an additional DNN model is trained which evaluates the local decisions of the local subnetworks and generates a final, global decision. With respect to the analogy to DDMs, the DNN models can be loosely interpreted as a coarse problem and hence, the new approach can be interpreted as a two-level domain decomposition. In this paper, we apply the proposed architecture to image classification problems using CNNs. Experimental results for different two-dimensional image classification problems are provided, as well as a face recognition problem and a classification problem for three-dimensional computed tomography (CT) scans. Therefore, classical Residual Network (ResNet) and VGG architectures are considered. More modern architectures, such as, e.g., MobileNet2, are left for future work. The results show that the proposed approach can significantly accelerate the required training time compared to the global model and, additionally, can also help to improve the accuracy of the underlying classification problem.
{"title":"A Domain Decomposition–Based CNN-DNN Architecture for Model Parallel Training Applied to Image Recognition Problems","authors":"Axel Klawonn, Martin Lanser, Janine Weber","doi":"10.1137/23m1562202","DOIUrl":"https://doi.org/10.1137/23m1562202","url":null,"abstract":"SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page C557-C582, October 2024. <br/> Abstract. Deep neural networks (DNNs) and, in particular, convolutional neural networks (CNNs) have brought significant advances in a wide range of modern computer application problems. However, the increasing availability of large numbers of datasets and the increasing available computational power of modern computers have led to steady growth in the complexity and size of DNN and CNN models, respectively, and thus, to longer training times. Hence, various methods and attempts have been developed to accelerate and parallelize the training of complex network architectures. In this work, a novel CNN-DNN architecture is proposed that naturally supports a model parallel training strategy and that is loosely inspired by two-level domain decomposition methods (DDMs). First, local CNN models, that is, subnetworks, are defined that operate on overlapping or nonoverlapping parts of the input data, for example, subimages. The subnetworks can be trained completely in parallel and independently of each other. Each subnetwork then outputs a local decision for the given machine learning problem which is exclusively based on the respective local input data. Subsequently, in a second step, an additional DNN model is trained which evaluates the local decisions of the local subnetworks and generates a final, global decision. With respect to the analogy to DDMs, the DNN models can be loosely interpreted as a coarse problem and hence, the new approach can be interpreted as a two-level domain decomposition. In this paper, we apply the proposed architecture to image classification problems using CNNs. Experimental results for different two-dimensional image classification problems are provided, as well as a face recognition problem and a classification problem for three-dimensional computed tomography (CT) scans. Therefore, classical Residual Network (ResNet) and VGG architectures are considered. More modern architectures, such as, e.g., MobileNet2, are left for future work. The results show that the proposed approach can significantly accelerate the required training time compared to the global model and, additionally, can also help to improve the accuracy of the underlying classification problem.","PeriodicalId":49526,"journal":{"name":"SIAM Journal on Scientific Computing","volume":"75 1 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142217595","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tareq Zaman, Nicolas Nytko, Ali Taghibakhshi, Scott MacLachlan, Luke Olson, Matthew West
SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page A2819-A2847, October 2024. Abstract. Clustering is a commonplace problem in many areas of data science, with applications in biology and bioinformatics, understanding chemical structure, image segmentation, building recommender systems, and many more fields. While there are many different clustering variants (based on given distance or graph structure, probability distributions, or data density), we consider here the problem of clustering nodes in a graph, motivated by the problem of aggregating discrete degrees of freedom in multigrid and domain decomposition methods for solving sparse linear systems. Specifically, we consider the challenge of forming balanced clusters in the graph of a sparse matrix for use in algebraic multigrid, although the algorithm has general applicability. Based on an extension of the Bellman–Ford algorithm, we generalize Lloyd’s algorithm for partitioning subsets of [math] to balance the number of nodes in each cluster; this is accompanied by a rebalancing algorithm that reduces the overall energy in the system. The algorithm provides control over the number of clusters and leads to “well centered” partitions of the graph. Theoretical results are provided to establish linear complexity and numerical results in the context of algebraic multigrid highlight the benefits of improved clustering. Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: Code and data available” as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/lukeolson/paper-lloyd-data and in the supplementary materials (paper-lloyd-data.zip [88.1MB]).
{"title":"Generalizing Lloyd’s Algorithm for Graph Clustering","authors":"Tareq Zaman, Nicolas Nytko, Ali Taghibakhshi, Scott MacLachlan, Luke Olson, Matthew West","doi":"10.1137/23m1556800","DOIUrl":"https://doi.org/10.1137/23m1556800","url":null,"abstract":"SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page A2819-A2847, October 2024. <br/> Abstract. Clustering is a commonplace problem in many areas of data science, with applications in biology and bioinformatics, understanding chemical structure, image segmentation, building recommender systems, and many more fields. While there are many different clustering variants (based on given distance or graph structure, probability distributions, or data density), we consider here the problem of clustering nodes in a graph, motivated by the problem of aggregating discrete degrees of freedom in multigrid and domain decomposition methods for solving sparse linear systems. Specifically, we consider the challenge of forming balanced clusters in the graph of a sparse matrix for use in algebraic multigrid, although the algorithm has general applicability. Based on an extension of the Bellman–Ford algorithm, we generalize Lloyd’s algorithm for partitioning subsets of [math] to balance the number of nodes in each cluster; this is accompanied by a rebalancing algorithm that reduces the overall energy in the system. The algorithm provides control over the number of clusters and leads to “well centered” partitions of the graph. Theoretical results are provided to establish linear complexity and numerical results in the context of algebraic multigrid highlight the benefits of improved clustering. Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: Code and data available” as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/lukeolson/paper-lloyd-data and in the supplementary materials (paper-lloyd-data.zip [88.1MB]).","PeriodicalId":49526,"journal":{"name":"SIAM Journal on Scientific Computing","volume":"19 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142217598","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page C535-C556, October 2024. Abstract. We present a neural network approach for approximating the value function of high-dimensional stochastic control problems. Our training process simultaneously updates our value function estimate and identifies the part of the state space likely to be visited by optimal trajectories. Our approach leverages insights from optimal control theory and the fundamental relation between semilinear parabolic partial differential equations and forward-backward stochastic differential equations. To focus the sampling on relevant states during neural network training, we use the stochastic Pontryagin maximum principle (PMP) to obtain the optimal controls for the current value function estimate. By design, our approach coincides with the method of characteristics for the nonviscous Hamilton–Jacobi–Bellman equation arising in deterministic control problems. Our training loss consists of a weighted sum of the objective functional of the control problem and penalty terms that enforce the HJB equations along the sampled trajectories. Importantly, training is unsupervised in that it does not require solutions of the control problem. Our numerical experiments highlight our scheme’s ability to identify the relevant parts of the state space and produce meaningful value estimates. Using a two-dimensional model problem, we demonstrate the importance of the stochastic PMP to inform the sampling and compare it to a finite element approach. With a nonlinear control affine quadcopter example, we illustrate that our approach can handle complicated dynamics. For a 100-dimensional benchmark problem, we demonstrate that our approach improves accuracy and time-to-solution, and, via a modification, we show the wider applicability of our scheme. Reproducibility of computational results.This paper has been awarded the “SIAM Reproducibility Badge: Code and data available” as recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/EmoryMLIP/NeuralSOC and in the supplementary material (NeuralSOC-main.zip [ 29.9MB]).
{"title":"A Neural Network Approach for Stochastic Optimal Control","authors":"Xingjian Li, Deepanshu Verma, Lars Ruthotto","doi":"10.1137/23m155832x","DOIUrl":"https://doi.org/10.1137/23m155832x","url":null,"abstract":"SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page C535-C556, October 2024. <br/> Abstract. We present a neural network approach for approximating the value function of high-dimensional stochastic control problems. Our training process simultaneously updates our value function estimate and identifies the part of the state space likely to be visited by optimal trajectories. Our approach leverages insights from optimal control theory and the fundamental relation between semilinear parabolic partial differential equations and forward-backward stochastic differential equations. To focus the sampling on relevant states during neural network training, we use the stochastic Pontryagin maximum principle (PMP) to obtain the optimal controls for the current value function estimate. By design, our approach coincides with the method of characteristics for the nonviscous Hamilton–Jacobi–Bellman equation arising in deterministic control problems. Our training loss consists of a weighted sum of the objective functional of the control problem and penalty terms that enforce the HJB equations along the sampled trajectories. Importantly, training is unsupervised in that it does not require solutions of the control problem. Our numerical experiments highlight our scheme’s ability to identify the relevant parts of the state space and produce meaningful value estimates. Using a two-dimensional model problem, we demonstrate the importance of the stochastic PMP to inform the sampling and compare it to a finite element approach. With a nonlinear control affine quadcopter example, we illustrate that our approach can handle complicated dynamics. For a 100-dimensional benchmark problem, we demonstrate that our approach improves accuracy and time-to-solution, and, via a modification, we show the wider applicability of our scheme. Reproducibility of computational results.This paper has been awarded the “SIAM Reproducibility Badge: Code and data available” as recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/EmoryMLIP/NeuralSOC and in the supplementary material (NeuralSOC-main.zip [ 29.9MB]).","PeriodicalId":49526,"journal":{"name":"SIAM Journal on Scientific Computing","volume":"26 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142217597","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Scientific Computing, Ahead of Print. Abstract. In this study, we introduce algorithms optimized for GPU architectures, aimed at efficiently solving large sparse linear systems, a central challenge in Navier–Stokes pressure projection problems. Our approach includes an adaptation of the GMRES algorithm, drawing inspiration from the merged vector operations first proposed by Bielich et al. [Parallel Comput., 112 (2022), 102940]. This adaptation increases computational intensity on GPU platforms through optimized vector update strategies. The algorithm incorporates modified and classical Gram–Schmidt methods with an algebraic multigrid (AMG) preconditioner, each tailored for GPU performance. A key innovation in our work is the development of a Gram–Schmidt projector [math] employing a rank-1 perturbation of the identity matrix. Designed to maximize the high memory bandwidth utilization of the AMD MI-250X GPU, this approach includes a strategy for treating the unit diagonal that minimizes memory reads, leading to a 25% increase in computational efficiency. The application of perturbation theory further ensures that orthogonality loss is limited to [math], where [math] is the number of iterations. Additionally, we introduce a mixed AMG [math]-cycle strategy combining ILU(0) and [math]-Jacobi smoothers, which achieves a 30–50% reduction in GPU compute times compared to conventional methods, while maintaining low backward error. This strategy, alongside our novel treatment of the diagonal in triangular matrices, marks a substantial increase in AMG efficicency for GPU systems. We believe that these contributions represent a significant advance in optimizing GMRES+AMG algorithms for GPU computations. The empirical results demonstrate notable speed increments and maintain rigorous backward error bounds, underscoring the potential of our methods to substantially increase computational efficiency in large-scale scientific applications.
{"title":"Efficient GMRES+AMG on GPUs: Composite Smoothers And Mixed [math]-Cycles","authors":"Stephen Thomas, Allison H. Baker","doi":"10.1137/23m1578632","DOIUrl":"https://doi.org/10.1137/23m1578632","url":null,"abstract":"SIAM Journal on Scientific Computing, Ahead of Print. <br/> Abstract. In this study, we introduce algorithms optimized for GPU architectures, aimed at efficiently solving large sparse linear systems, a central challenge in Navier–Stokes pressure projection problems. Our approach includes an adaptation of the GMRES algorithm, drawing inspiration from the merged vector operations first proposed by Bielich et al. [Parallel Comput., 112 (2022), 102940]. This adaptation increases computational intensity on GPU platforms through optimized vector update strategies. The algorithm incorporates modified and classical Gram–Schmidt methods with an algebraic multigrid (AMG) preconditioner, each tailored for GPU performance. A key innovation in our work is the development of a Gram–Schmidt projector [math] employing a rank-1 perturbation of the identity matrix. Designed to maximize the high memory bandwidth utilization of the AMD MI-250X GPU, this approach includes a strategy for treating the unit diagonal that minimizes memory reads, leading to a 25% increase in computational efficiency. The application of perturbation theory further ensures that orthogonality loss is limited to [math], where [math] is the number of iterations. Additionally, we introduce a mixed AMG [math]-cycle strategy combining ILU(0) and [math]-Jacobi smoothers, which achieves a 30–50% reduction in GPU compute times compared to conventional methods, while maintaining low backward error. This strategy, alongside our novel treatment of the diagonal in triangular matrices, marks a substantial increase in AMG efficicency for GPU systems. We believe that these contributions represent a significant advance in optimizing GMRES+AMG algorithms for GPU computations. The empirical results demonstrate notable speed increments and maintain rigorous backward error bounds, underscoring the potential of our methods to substantially increase computational efficiency in large-scale scientific applications.","PeriodicalId":49526,"journal":{"name":"SIAM Journal on Scientific Computing","volume":"58 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142217596","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
E-M. El Arar, D. Sohier, P. de Oliveira Castro, E. Petit
SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page B579-B599, October 2024. Abstract. The main objective of this work is to investigate nonlinear errors and pairwise summation using stochastic rounding (SR) in variance computation algorithms. We estimate the forward error of computations under SR through two methods: the first is based on a bound of the variance and the Bienaymé–Chebyshev inequality, while the second is based on martingales and the Azuma–Hoeffding inequality. The study shows that for pairwise summation, using SR results in a probabilistic bound of the forward error proportional to [math] rather than the deterministic bound in [math] when using the default rounding mode. We examine two algorithms that compute the variance, one called “textbook” and the other “two-pass,” which both exhibit nonlinear errors. Using the two methods mentioned above, we show that the forward errors of these algorithms have probabilistic bounds under SR in [math] instead of [math] for the deterministic bounds. We show that this advantage holds using pairwise summation for both textbook and two-pass, with probabilistic bounds of the forward error proportional to [math]. Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: Code and data available” as recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow the reader to reproduce the results in this paper are available at https://github.com/verificarlo/sr-non-linear-bounds and in the supplementary material (sr-non-linear-bounds-main.zip [8.62KB]).
{"title":"Bounds on Nonlinear Errors for Variance Computation with Stochastic Rounding","authors":"E-M. El Arar, D. Sohier, P. de Oliveira Castro, E. Petit","doi":"10.1137/23m1563001","DOIUrl":"https://doi.org/10.1137/23m1563001","url":null,"abstract":"SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page B579-B599, October 2024. <br/> Abstract. The main objective of this work is to investigate nonlinear errors and pairwise summation using stochastic rounding (SR) in variance computation algorithms. We estimate the forward error of computations under SR through two methods: the first is based on a bound of the variance and the Bienaymé–Chebyshev inequality, while the second is based on martingales and the Azuma–Hoeffding inequality. The study shows that for pairwise summation, using SR results in a probabilistic bound of the forward error proportional to [math] rather than the deterministic bound in [math] when using the default rounding mode. We examine two algorithms that compute the variance, one called “textbook” and the other “two-pass,” which both exhibit nonlinear errors. Using the two methods mentioned above, we show that the forward errors of these algorithms have probabilistic bounds under SR in [math] instead of [math] for the deterministic bounds. We show that this advantage holds using pairwise summation for both textbook and two-pass, with probabilistic bounds of the forward error proportional to [math]. Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: Code and data available” as recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow the reader to reproduce the results in this paper are available at https://github.com/verificarlo/sr-non-linear-bounds and in the supplementary material (sr-non-linear-bounds-main.zip [8.62KB]).","PeriodicalId":49526,"journal":{"name":"SIAM Journal on Scientific Computing","volume":"72 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142217603","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page A2799-A2818, October 2024. Abstract. This work deals with an inverse source problem for the biharmonic wave equation. A two-stage numerical method is proposed to identify the unknown source from the multifrequency phaseless data. In the first stage, we introduce some artificial auxiliary point sources to the inverse source system and establish a phase retrieval formula. Theoretically, we point out that the phase can be uniquely determined and estimate the stability of this phase retrieval approach. Once the phase information is retrieved, the Fourier method is adopted to reconstruct the source function from the phased multifrequency data. The proposed method is easy to implement and there is no forward solver involved in the reconstruction. Numerical experiments are conducted to verify the performance of the proposed method.
{"title":"Inverse Source Problem of the Biharmonic Equation from Multifrequency Phaseless Data","authors":"Yan Chang, Yukun Guo, Yue Zhao","doi":"10.1137/24m162889x","DOIUrl":"https://doi.org/10.1137/24m162889x","url":null,"abstract":"SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page A2799-A2818, October 2024. <br/> Abstract. This work deals with an inverse source problem for the biharmonic wave equation. A two-stage numerical method is proposed to identify the unknown source from the multifrequency phaseless data. In the first stage, we introduce some artificial auxiliary point sources to the inverse source system and establish a phase retrieval formula. Theoretically, we point out that the phase can be uniquely determined and estimate the stability of this phase retrieval approach. Once the phase information is retrieved, the Fourier method is adopted to reconstruct the source function from the phased multifrequency data. The proposed method is easy to implement and there is no forward solver involved in the reconstruction. Numerical experiments are conducted to verify the performance of the proposed method.","PeriodicalId":49526,"journal":{"name":"SIAM Journal on Scientific Computing","volume":"163 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142217602","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Minglei Yang, Pengjun Wang, Diego del-Castillo-Negrete, Yanzhao Cao, Guannan Zhang
SIAM Journal on Scientific Computing, Volume 46, Issue 4, Page C508-C533, August 2024. Abstract. We present a pseudoreversible normalizing flow method for efficiently generating samples of the state of a stochastic differential equation (SDE) with various initial distributions. The primary objective is to construct an accurate and efficient sampler that can be used as a surrogate model for computationally expensive numerical integration of SDEs, such as those employed in particle simulation. After training, the normalizing flow model can directly generate samples of the SDE’s final state without simulating trajectories. The existing normalizing flow model for SDEs depends on the initial distribution, meaning the model needs to be retrained when the initial distribution changes. The main novelty of our normalizing flow model is that it can learn the conditional distribution of the state, i.e., the distribution of the final state conditional on any initial state, such that the model only needs to be trained once and the trained model can be used to handle various initial distributions. This feature can provide a significant computational saving in studies of how the final state varies with the initial distribution. Additionally, we propose to use a pseudoreversible network architecture to define the normalizing flow model, which has sufficient expressive power and training efficiency for a variety of SDEs in science and engineering, e.g., in particle physics. We provide a rigorous convergence analysis of the pseudoreversible normalizing flow model to the target probability density function in the Kullback–Leibler divergence metric. Numerical experiments are provided to demonstrate the effectiveness of the proposed normalizing flow model. Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: Code and data available” as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/mlmathphy/PRNF and in the supplementary materials (PRNF-main.zip [27.4MB]).
{"title":"A Pseudoreversible Normalizing Flow for Stochastic Dynamical Systems with Various Initial Distributions","authors":"Minglei Yang, Pengjun Wang, Diego del-Castillo-Negrete, Yanzhao Cao, Guannan Zhang","doi":"10.1137/23m1585635","DOIUrl":"https://doi.org/10.1137/23m1585635","url":null,"abstract":"SIAM Journal on Scientific Computing, Volume 46, Issue 4, Page C508-C533, August 2024. <br/> Abstract. We present a pseudoreversible normalizing flow method for efficiently generating samples of the state of a stochastic differential equation (SDE) with various initial distributions. The primary objective is to construct an accurate and efficient sampler that can be used as a surrogate model for computationally expensive numerical integration of SDEs, such as those employed in particle simulation. After training, the normalizing flow model can directly generate samples of the SDE’s final state without simulating trajectories. The existing normalizing flow model for SDEs depends on the initial distribution, meaning the model needs to be retrained when the initial distribution changes. The main novelty of our normalizing flow model is that it can learn the conditional distribution of the state, i.e., the distribution of the final state conditional on any initial state, such that the model only needs to be trained once and the trained model can be used to handle various initial distributions. This feature can provide a significant computational saving in studies of how the final state varies with the initial distribution. Additionally, we propose to use a pseudoreversible network architecture to define the normalizing flow model, which has sufficient expressive power and training efficiency for a variety of SDEs in science and engineering, e.g., in particle physics. We provide a rigorous convergence analysis of the pseudoreversible normalizing flow model to the target probability density function in the Kullback–Leibler divergence metric. Numerical experiments are provided to demonstrate the effectiveness of the proposed normalizing flow model. Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: Code and data available” as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/mlmathphy/PRNF and in the supplementary materials (PRNF-main.zip [27.4MB]).","PeriodicalId":49526,"journal":{"name":"SIAM Journal on Scientific Computing","volume":"7 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142227246","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Scientific Computing, Volume 46, Issue 4, Page A2774-A2797, August 2024. Abstract. A sketch-and-select Arnoldi process to generate a well-conditioned basis of a Krylov space at low cost is proposed. At each iteration the procedure utilizes randomized sketching to select a limited number of previously computed basis vectors to project out of the current basis vector. The computational cost grows linearly with the dimension of the Krylov space. The subset selection problem for the projection step is approximately solved with a number of heuristic algorithms and greedy methods used in statistical learning and compressive sensing. Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: Code and data available” as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/simunec/sketch-select-arnoldi and in the supplementary materials (sketch-select-arnoldi-main.zip [2.21MB]).
{"title":"A Sketch-and-Select Arnoldi Process","authors":"Stefan Güttel, Igor Simunec","doi":"10.1137/23m1588007","DOIUrl":"https://doi.org/10.1137/23m1588007","url":null,"abstract":"SIAM Journal on Scientific Computing, Volume 46, Issue 4, Page A2774-A2797, August 2024. <br/> Abstract. A sketch-and-select Arnoldi process to generate a well-conditioned basis of a Krylov space at low cost is proposed. At each iteration the procedure utilizes randomized sketching to select a limited number of previously computed basis vectors to project out of the current basis vector. The computational cost grows linearly with the dimension of the Krylov space. The subset selection problem for the projection step is approximately solved with a number of heuristic algorithms and greedy methods used in statistical learning and compressive sensing. Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: Code and data available” as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/simunec/sketch-select-arnoldi and in the supplementary materials (sketch-select-arnoldi-main.zip [2.21MB]).","PeriodicalId":49526,"journal":{"name":"SIAM Journal on Scientific Computing","volume":"63 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142227247","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Christoph Reisinger, Wolfgang Stockinger, Yufei Zhang
SIAM Journal on Scientific Computing, Volume 46, Issue 4, Page A2737-A2773, August 2024. Abstract. We propose a PDE-based accelerated gradient algorithm for optimal feedback controls of McKean–Vlasov dynamics that involve mean-field interactions both in the state and action. The method exploits a forward-backward splitting approach and iteratively refines the approximate controls based on the gradients of smooth costs, the proximal maps of nonsmooth costs, and dynamically updated momentum parameters. At each step, the state dynamics is approximated via a particle system, and the required gradient is evaluated through a coupled system of nonlocal linear PDEs. The latter is solved by finite difference approximation or neural network-based residual approximation, depending on the state dimension. We present exhaustive numerical experiments for low- and high-dimensional mean-field control problems, including sparse stabilization of stochastic Cucker–Smale models, which reveal that our algorithm captures important structures of the optimal feedback control and achieves a robust performance with respect to parameter perturbation.
{"title":"A Fast Iterative PDE-Based Algorithm for Feedback Controls of Nonsmooth Mean-Field Control Problems","authors":"Christoph Reisinger, Wolfgang Stockinger, Yufei Zhang","doi":"10.1137/21m1441158","DOIUrl":"https://doi.org/10.1137/21m1441158","url":null,"abstract":"SIAM Journal on Scientific Computing, Volume 46, Issue 4, Page A2737-A2773, August 2024. <br/> Abstract. We propose a PDE-based accelerated gradient algorithm for optimal feedback controls of McKean–Vlasov dynamics that involve mean-field interactions both in the state and action. The method exploits a forward-backward splitting approach and iteratively refines the approximate controls based on the gradients of smooth costs, the proximal maps of nonsmooth costs, and dynamically updated momentum parameters. At each step, the state dynamics is approximated via a particle system, and the required gradient is evaluated through a coupled system of nonlocal linear PDEs. The latter is solved by finite difference approximation or neural network-based residual approximation, depending on the state dimension. We present exhaustive numerical experiments for low- and high-dimensional mean-field control problems, including sparse stabilization of stochastic Cucker–Smale models, which reveal that our algorithm captures important structures of the optimal feedback control and achieves a robust performance with respect to parameter perturbation.","PeriodicalId":49526,"journal":{"name":"SIAM Journal on Scientific Computing","volume":"23 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142217604","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号