Gauss proposed the problem of how to enumerate the number of solutions for placing $N$ queens on an $Ntimes N$ chess board, so no two queens attack each other. The N-queen problem is a classic problem in combinatorics. We describe a variety of Monte Carlo (MC) methods for counting the number of solutions. In particular, we propose a quantile re-ordering based on the Lorenz curve of a sum that is related to counting the number of solutions. We show his approach leads to an efficient polynomial-time solution. Other MC methods include vertical likelihood Monte Carlo, importance sampling, slice sampling, simulated annealing, energy-level sampling, and nested-sampling. Sampling binary matrices that identify the locations of the queens on the board can be done with a Swendsen-Wang style algorithm. Our Monte Carlo approach counts the number of solutions in polynomial time.
高斯提出了这样一个问题:如何枚举出在一个 N 次 N 元的棋盘上摆放 N 个皇后的解的个数,从而避免两个皇后互相攻击。N 皇后问题是组合数学中的一个经典问题。我们介绍了各种计算解数的蒙特卡罗(MC)方法。特别是,我们提出了一种基于洛伦兹曲线的量子重排序方法,它与计算解的数量有关。我们证明了他的方法能带来高效的多项式时间解决方案。其他 MC 方法包括理论似然蒙特卡罗、重要性采样、切片采样、模拟嵌套、能量级采样和嵌套采样。对确定棋盘上皇后位置的二进制矩阵进行采样,可采用斯文森-旺(Swendsen-Wang)式算法。我们的蒙特卡罗方法可以在多项式时间内计算出解决方案的数量。
{"title":"Counting $N$ Queens","authors":"Nick Polson, Vadim Sokolov","doi":"arxiv-2407.08830","DOIUrl":"https://doi.org/arxiv-2407.08830","url":null,"abstract":"Gauss proposed the problem of how to enumerate the number of solutions for\u0000placing $N$ queens on an $Ntimes N$ chess board, so no two queens attack each\u0000other. The N-queen problem is a classic problem in combinatorics. We describe a\u0000variety of Monte Carlo (MC) methods for counting the number of solutions. In\u0000particular, we propose a quantile re-ordering based on the Lorenz curve of a\u0000sum that is related to counting the number of solutions. We show his approach\u0000leads to an efficient polynomial-time solution. Other MC methods include\u0000vertical likelihood Monte Carlo, importance sampling, slice sampling, simulated\u0000annealing, energy-level sampling, and nested-sampling. Sampling binary matrices\u0000that identify the locations of the queens on the board can be done with a\u0000Swendsen-Wang style algorithm. Our Monte Carlo approach counts the number of\u0000solutions in polynomial time.","PeriodicalId":501215,"journal":{"name":"arXiv - STAT - Computation","volume":"25 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141720746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Håvard HegrePeace Research Institute OsloDepartment of Peace and Conflict Research, Uppsala University, Paola VescoPeace Research Institute OsloDepartment of Peace and Conflict Research, Uppsala University, Michael ColaresiDepartment of Peace and Conflict Research, Uppsala UniversityUniversity of Pittsburgh, Jonas VestbyPeace Research Institute Oslo, Alexa TimlickPeace Research Institute Oslo, Noorain Syed KazmiPeace Research Institute Oslo, Friederike BeckerInstitute of Statistics, Marco BinettiCenter for Crisis Early Warning, University of the Bundeswehr Munich, Tobias BodentienInstitute of Statistics, Tobias BohneCenter for Crisis Early Warning, University of the Bundeswehr Munich, Patrick T. BrandtSchool of Economic, Political, and Policy Sciences, University of Texas, Dallas, Thomas ChadefauxTrinity College Dublin, Simon DrauzInstitute of Statistics, Christoph DworschakUniversity of York, Vito D'OrazioWest Virginia University, Cornelius FritzPennsylvania State University, Hannah FrankTrinity College Dublin, Kristian Skrede GleditschUniversity of EssexPeace Research Institute Oslo, Sonja HäffnerCenter for Crisis Early Warning, University of the Bundeswehr Munich, Martin HoferUniversity College London, Finn L. KlebeUniversity College London, Luca MacisDepartment of Economics and Statistics Cognetti de Martiis, University of Turin, Alexandra MalagaInstitute for Economic Analysis, Barcelona, Marius MehrlUniversity of Leeds, Nils W. MetternichUniversity College London, Daniel MittermaierCenter for Crisis Early Warning, University of the Bundeswehr Munich, David MuchlinskiGeorgia Tech, Hannes MuellerInstitute for Economic Analysis, BarcelonaBarcelona School of Economics, Christian OswaldCenter for Crisis Early Warning, University of the Bundeswehr Munich, Paola PisanoDepartment of Economics and Statistics Cognetti de Martiis, University of Turin, David RandahlDepartment of Peace and Conflict Research, Uppsala University, Christopher RauhUniversity of Cambridge, Lotta RüterInstitute of Statistics, Thomas SchincariolTrinity College Dublin, Benjamin SeimonFundació Economia Analitica, Elena SilettiDepartment of Economics and Statistics Cognetti de Martiis, University of Turin, Marco TagliapietraDepartment of Economics and Statistics Cognetti de Martiis, University of Turin, Chandler ThornhillGeorgia Tech, Johan VegeliusDepartment of Medical Sciences, Uppsala University, Julian WalterskirchenCenter for Crisis Early Warning, University of the Bundeswehr Munich
This draft article outlines a prediction challenge where the target is to forecast the number of fatalities in armed conflicts, in the form of the UCDP `best' estimates, aggregated to the VIEWS units of analysis. It presents the format of the contributions, the evaluation metric, and the procedures, and a brief summary of the contributions. The article serves a function analogous to a pre-analysis plan: a statement of the forecasting models made publicly available before the true future prediction window commences. More information on the challenge, and all data referred to in this document, can be found at https://viewsforecasting.org/research/prediction-challenge-2023.
{"title":"The 2023/24 VIEWS Prediction Challenge: Predicting the Number of Fatalities in Armed Conflict, with Uncertainty","authors":"Håvard HegrePeace Research Institute OsloDepartment of Peace and Conflict Research, Uppsala University, Paola VescoPeace Research Institute OsloDepartment of Peace and Conflict Research, Uppsala University, Michael ColaresiDepartment of Peace and Conflict Research, Uppsala UniversityUniversity of Pittsburgh, Jonas VestbyPeace Research Institute Oslo, Alexa TimlickPeace Research Institute Oslo, Noorain Syed KazmiPeace Research Institute Oslo, Friederike BeckerInstitute of Statistics, Marco BinettiCenter for Crisis Early Warning, University of the Bundeswehr Munich, Tobias BodentienInstitute of Statistics, Tobias BohneCenter for Crisis Early Warning, University of the Bundeswehr Munich, Patrick T. BrandtSchool of Economic, Political, and Policy Sciences, University of Texas, Dallas, Thomas ChadefauxTrinity College Dublin, Simon DrauzInstitute of Statistics, Christoph DworschakUniversity of York, Vito D'OrazioWest Virginia University, Cornelius FritzPennsylvania State University, Hannah FrankTrinity College Dublin, Kristian Skrede GleditschUniversity of EssexPeace Research Institute Oslo, Sonja HäffnerCenter for Crisis Early Warning, University of the Bundeswehr Munich, Martin HoferUniversity College London, Finn L. KlebeUniversity College London, Luca MacisDepartment of Economics and Statistics Cognetti de Martiis, University of Turin, Alexandra MalagaInstitute for Economic Analysis, Barcelona, Marius MehrlUniversity of Leeds, Nils W. MetternichUniversity College London, Daniel MittermaierCenter for Crisis Early Warning, University of the Bundeswehr Munich, David MuchlinskiGeorgia Tech, Hannes MuellerInstitute for Economic Analysis, BarcelonaBarcelona School of Economics, Christian OswaldCenter for Crisis Early Warning, University of the Bundeswehr Munich, Paola PisanoDepartment of Economics and Statistics Cognetti de Martiis, University of Turin, David RandahlDepartment of Peace and Conflict Research, Uppsala University, Christopher RauhUniversity of Cambridge, Lotta RüterInstitute of Statistics, Thomas SchincariolTrinity College Dublin, Benjamin SeimonFundació Economia Analitica, Elena SilettiDepartment of Economics and Statistics Cognetti de Martiis, University of Turin, Marco TagliapietraDepartment of Economics and Statistics Cognetti de Martiis, University of Turin, Chandler ThornhillGeorgia Tech, Johan VegeliusDepartment of Medical Sciences, Uppsala University, Julian WalterskirchenCenter for Crisis Early Warning, University of the Bundeswehr Munich","doi":"arxiv-2407.11045","DOIUrl":"https://doi.org/arxiv-2407.11045","url":null,"abstract":"This draft article outlines a prediction challenge where the target is to\u0000forecast the number of fatalities in armed conflicts, in the form of the UCDP\u0000`best' estimates, aggregated to the VIEWS units of analysis. It presents the\u0000format of the contributions, the evaluation metric, and the procedures, and a\u0000brief summary of the contributions. The article serves a function analogous to\u0000a pre-analysis plan: a statement of the forecasting models made publicly\u0000available before the true future prediction window commences. More information\u0000on the challenge, and all data referred to in this document, can be found at\u0000https://viewsforecasting.org/research/prediction-challenge-2023.","PeriodicalId":501215,"journal":{"name":"arXiv - STAT - Computation","volume":"50 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141720747","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Måns Magnusson, Jakob Torgander, Paul-Christian Bürkner, Lu Zhang, Bob Carpenter, Aki Vehtari
The generality and robustness of inference algorithms is critical to the success of widely used probabilistic programming languages such as Stan, PyMC, Pyro, and Turing.jl. When designing a new general-purpose inference algorithm, whether it involves Monte Carlo sampling or variational approximation, the fundamental problem arises in evaluating its accuracy and efficiency across a range of representative target models. To solve this problem, we propose posteriordb, a database of models and data sets defining target densities along with reference Monte Carlo draws. We further provide a guide to the best practices in using posteriordb for model evaluation and comparison. To provide a wide range of realistic target densities, posteriordb currently comprises 120 representative models and has been instrumental in developing several general inference algorithms.
{"title":"posteriordb: Testing, Benchmarking and Developing Bayesian Inference Algorithms","authors":"Måns Magnusson, Jakob Torgander, Paul-Christian Bürkner, Lu Zhang, Bob Carpenter, Aki Vehtari","doi":"arxiv-2407.04967","DOIUrl":"https://doi.org/arxiv-2407.04967","url":null,"abstract":"The generality and robustness of inference algorithms is critical to the\u0000success of widely used probabilistic programming languages such as Stan, PyMC,\u0000Pyro, and Turing.jl. When designing a new general-purpose inference algorithm,\u0000whether it involves Monte Carlo sampling or variational approximation, the\u0000fundamental problem arises in evaluating its accuracy and efficiency across a\u0000range of representative target models. To solve this problem, we propose\u0000posteriordb, a database of models and data sets defining target densities along\u0000with reference Monte Carlo draws. We further provide a guide to the best\u0000practices in using posteriordb for model evaluation and comparison. To provide\u0000a wide range of realistic target densities, posteriordb currently comprises 120\u0000representative models and has been instrumental in developing several general\u0000inference algorithms.","PeriodicalId":501215,"journal":{"name":"arXiv - STAT - Computation","volume":"40 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141574477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We introduce a fast algorithm for Gaussian process regression in low dimensions, applicable to a widely-used family of non-stationary kernels. The non-stationarity of these kernels is induced by arbitrary spatially-varying vertical and horizontal scales. In particular, any stationary kernel can be accommodated as a special case, and we focus especially on the generalization of the standard Mat'ern kernel. Our subroutine for kernel matrix-vector multiplications scales almost optimally as $O(Nlog N)$, where $N$ is the number of regression points. Like the recently developed equispaced Fourier Gaussian process (EFGP) methodology, which is applicable only to stationary kernels, our approach exploits non-uniform fast Fourier transforms (NUFFTs). We offer a complete analysis controlling the approximation error of our method, and we validate the method's practical performance with numerical experiments. In particular we demonstrate improved scalability compared to to state-of-the-art rank-structured approaches in spatial dimension $d>1$.
{"title":"Gaussian process regression with log-linear scaling for common non-stationary kernels","authors":"P. Michael Kielstra, Michael Lindsey","doi":"arxiv-2407.03608","DOIUrl":"https://doi.org/arxiv-2407.03608","url":null,"abstract":"We introduce a fast algorithm for Gaussian process regression in low\u0000dimensions, applicable to a widely-used family of non-stationary kernels. The\u0000non-stationarity of these kernels is induced by arbitrary spatially-varying\u0000vertical and horizontal scales. In particular, any stationary kernel can be\u0000accommodated as a special case, and we focus especially on the generalization\u0000of the standard Mat'ern kernel. Our subroutine for kernel matrix-vector\u0000multiplications scales almost optimally as $O(Nlog N)$, where $N$ is the\u0000number of regression points. Like the recently developed equispaced Fourier\u0000Gaussian process (EFGP) methodology, which is applicable only to stationary\u0000kernels, our approach exploits non-uniform fast Fourier transforms (NUFFTs). We\u0000offer a complete analysis controlling the approximation error of our method,\u0000and we validate the method's practical performance with numerical experiments.\u0000In particular we demonstrate improved scalability compared to to\u0000state-of-the-art rank-structured approaches in spatial dimension $d>1$.","PeriodicalId":501215,"journal":{"name":"arXiv - STAT - Computation","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141574479","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Cyrus Mostajeran, Nathaël Da Costa, Graham Van Goffrier, Rodolphe Sepulchre
We present a geometric framework for the processing of SPD-valued data that preserves subspace structures and is based on the efficient computation of extreme generalized eigenvalues. This is achieved through the use of the Thompson geometry of the semidefinite cone. We explore a particular geodesic space structure in detail and establish several properties associated with it. Finally, we review a novel inductive mean of SPD matrices based on this geometry.
我们提出了一个处理 SPD 值数据的几何框架,该框架保留了子空间结构,并以高效计算极端广义特征值为基础。这是通过使用半定锥的汤普森几何来实现的。我们详细探讨了一种特殊的大地空间结构,并建立了与之相关的几个属性。最后,我们回顾了基于这种几何的 SPD 矩阵的一种新颖的归纳平均值。
{"title":"Geometric statistics with subspace structure preservation for SPD matrices","authors":"Cyrus Mostajeran, Nathaël Da Costa, Graham Van Goffrier, Rodolphe Sepulchre","doi":"arxiv-2407.03382","DOIUrl":"https://doi.org/arxiv-2407.03382","url":null,"abstract":"We present a geometric framework for the processing of SPD-valued data that\u0000preserves subspace structures and is based on the efficient computation of\u0000extreme generalized eigenvalues. This is achieved through the use of the\u0000Thompson geometry of the semidefinite cone. We explore a particular geodesic\u0000space structure in detail and establish several properties associated with it.\u0000Finally, we review a novel inductive mean of SPD matrices based on this\u0000geometry.","PeriodicalId":501215,"journal":{"name":"arXiv - STAT - Computation","volume":"18 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141574481","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Niccolò Anceschi, Augusto Fasano, Beatrice Franzolini, Giovanni Rebaudo
Generalized linear models (GLMs) arguably represent the standard approach for statistical regression beyond the Gaussian likelihood scenario. When Bayesian formulations are employed, the general absence of a tractable posterior distribution has motivated the development of deterministic approximations, which are generally more scalable than sampling techniques. Among them, expectation propagation (EP) showed extreme accuracy, usually higher than many variational Bayes solutions. However, the higher computational cost of EP posed concerns about its practical feasibility, especially in high-dimensional settings. We address these concerns by deriving a novel efficient formulation of EP for GLMs, whose cost scales linearly in the number of covariates p. This reduces the state-of-the-art O(p^2 n) per-iteration computational cost of the EP routine for GLMs to O(p n min{p,n}), with n being the sample size. We also show that, for binary models and log-linear GLMs approximate predictive means can be obtained at no additional cost. To preserve efficient moment matching for count data, we propose employing a combination of log-normal Laplace transform approximations, avoiding numerical integration. These novel results open the possibility of employing EP in settings that were believed to be practically impossible. Improvements over state-of-the-art approaches are illustrated both for simulated and real data. The efficient EP implementation is available at https://github.com/niccoloanceschi/EPglm.
广义线性模型(GLM)可以说是超越高斯似然情景的标准统计回归方法。在使用贝叶斯公式时,由于普遍缺乏可操作的后分布,因此人们开发了确定性近似方法,这种方法通常比抽样技术更具可扩展性。其中,期望传播(EP)显示出极高的准确性,通常高于许多变量贝叶斯解决方案。然而,EP 较高的计算成本使人们对其实际可行性产生了担忧,尤其是在高维环境中。为了解决这些问题,我们为 GLMs 推导了一种新的高效 EP 方案,其成本与协方差的数量 p 成线性比例,从而将 GLMs 的 EP 例程的最新 O(p^2 n) 每次迭代计算成本降至 O(p,n),n 为样本大小。我们还证明,对于二元模型和对数线性 GLM,可以在不增加成本的情况下获得近似预测均值。为了保持计数数据的有效矩匹配,我们建议采用对数正态拉普变换近似的组合,避免数值积分。这些新颖的结果为在人们认为实际上不可能的情况下使用 EP 提供了可能性。在模拟数据和真实数据方面,与最先进的方法相比都有很大改进。高效的 EP 实现可在 https://github.com/niccoloanceschi/EPglm 上获取。
{"title":"Scalable expectation propagation for generalized linear models","authors":"Niccolò Anceschi, Augusto Fasano, Beatrice Franzolini, Giovanni Rebaudo","doi":"arxiv-2407.02128","DOIUrl":"https://doi.org/arxiv-2407.02128","url":null,"abstract":"Generalized linear models (GLMs) arguably represent the standard approach for\u0000statistical regression beyond the Gaussian likelihood scenario. When Bayesian\u0000formulations are employed, the general absence of a tractable posterior\u0000distribution has motivated the development of deterministic approximations,\u0000which are generally more scalable than sampling techniques. Among them,\u0000expectation propagation (EP) showed extreme accuracy, usually higher than many\u0000variational Bayes solutions. However, the higher computational cost of EP posed\u0000concerns about its practical feasibility, especially in high-dimensional\u0000settings. We address these concerns by deriving a novel efficient formulation\u0000of EP for GLMs, whose cost scales linearly in the number of covariates p. This\u0000reduces the state-of-the-art O(p^2 n) per-iteration computational cost of the\u0000EP routine for GLMs to O(p n min{p,n}), with n being the sample size. We also\u0000show that, for binary models and log-linear GLMs approximate predictive means\u0000can be obtained at no additional cost. To preserve efficient moment matching\u0000for count data, we propose employing a combination of log-normal Laplace\u0000transform approximations, avoiding numerical integration. These novel results\u0000open the possibility of employing EP in settings that were believed to be\u0000practically impossible. Improvements over state-of-the-art approaches are\u0000illustrated both for simulated and real data. The efficient EP implementation\u0000is available at https://github.com/niccoloanceschi/EPglm.","PeriodicalId":501215,"journal":{"name":"arXiv - STAT - Computation","volume":"21 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141531894","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We propose a general-purpose approximation to the Ferguson-Klass algorithm for generating samples from L'evy processes without Gaussian components. We show that the proposed method is more than 1000 times faster than the standard Ferguson-Klass algorithm without a significant loss of precision. This method can open an avenue for computationally efficient and scalable Bayesian nonparametric models which go beyond conjugacy assumptions, as demonstrated in the examples section.
{"title":"A General Purpose Approximation to the Ferguson-Klass Algorithm for Sampling from Lévy Processes Without Gaussian Components","authors":"Dawid Bernaciak, Jim E. Griffin","doi":"arxiv-2407.01483","DOIUrl":"https://doi.org/arxiv-2407.01483","url":null,"abstract":"We propose a general-purpose approximation to the Ferguson-Klass algorithm\u0000for generating samples from L'evy processes without Gaussian components. We\u0000show that the proposed method is more than 1000 times faster than the standard\u0000Ferguson-Klass algorithm without a significant loss of precision. This method\u0000can open an avenue for computationally efficient and scalable Bayesian\u0000nonparametric models which go beyond conjugacy assumptions, as demonstrated in\u0000the examples section.","PeriodicalId":501215,"journal":{"name":"arXiv - STAT - Computation","volume":"189 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141509193","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
For linear systems $Ax=b$ we develop iterative algorithms based on a sketch-and-project approach. By using judicious choices for the sketch, such as the history of residuals, we develop weighting strategies that enable short recursive formulas. The proposed algorithms have a low memory footprint and iteration complexity compared to regular sketch-and-project methods. In a set of numerical experiments the new methods compare well to GMRES, SYMMLQ and state-of-the-art randomized solvers.
{"title":"Structured Sketching for Linear Systems","authors":"Johannes J Brust, Michael A Saunders","doi":"arxiv-2407.00746","DOIUrl":"https://doi.org/arxiv-2407.00746","url":null,"abstract":"For linear systems $Ax=b$ we develop iterative algorithms based on a\u0000sketch-and-project approach. By using judicious choices for the sketch, such as\u0000the history of residuals, we develop weighting strategies that enable short\u0000recursive formulas. The proposed algorithms have a low memory footprint and\u0000iteration complexity compared to regular sketch-and-project methods. In a set\u0000of numerical experiments the new methods compare well to GMRES, SYMMLQ and\u0000state-of-the-art randomized solvers.","PeriodicalId":501215,"journal":{"name":"arXiv - STAT - Computation","volume":"20 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141531895","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Motivated by applications in emergency response and experimental design, we consider smooth stochastic optimization problems over probability measures supported on compact subsets of the Euclidean space. With the influence function as the variational object, we construct a deterministic Frank-Wolfe (dFW) recursion for probability spaces, made especially possible by a lemma that identifies a ``closed-form'' solution to the infinite-dimensional Frank-Wolfe sub-problem. Each iterate in dFW is expressed as a convex combination of the incumbent iterate and a Dirac measure concentrating on the minimum of the influence function at the incumbent iterate. To address common application contexts that have access only to Monte Carlo observations of the objective and influence function, we construct a stochastic Frank-Wolfe (sFW) variation that generates a random sequence of probability measures constructed using minima of increasingly accurate estimates of the influence function. We demonstrate that sFW's optimality gap sequence exhibits $O(k^{-1})$ iteration complexity almost surely and in expectation for smooth convex objectives, and $O(k^{-1/2})$ (in Frank-Wolfe gap) for smooth non-convex objectives. Furthermore, we show that an easy-to-implement fixed-step, fixed-sample version of (sFW) exhibits exponential convergence to $varepsilon$-optimality. We end with a central limit theorem on the observed objective values at the sequence of generated random measures. To further intuition, we include several illustrative examples with exact influence function calculations.
{"title":"Deterministic and Stochastic Frank-Wolfe Recursion on Probability Spaces","authors":"Di Yu, Shane G. Henderson, Raghu Pasupathy","doi":"arxiv-2407.00307","DOIUrl":"https://doi.org/arxiv-2407.00307","url":null,"abstract":"Motivated by applications in emergency response and experimental design, we\u0000consider smooth stochastic optimization problems over probability measures\u0000supported on compact subsets of the Euclidean space. With the influence\u0000function as the variational object, we construct a deterministic Frank-Wolfe\u0000(dFW) recursion for probability spaces, made especially possible by a lemma\u0000that identifies a ``closed-form'' solution to the infinite-dimensional\u0000Frank-Wolfe sub-problem. Each iterate in dFW is expressed as a convex\u0000combination of the incumbent iterate and a Dirac measure concentrating on the\u0000minimum of the influence function at the incumbent iterate. To address common\u0000application contexts that have access only to Monte Carlo observations of the\u0000objective and influence function, we construct a stochastic Frank-Wolfe (sFW)\u0000variation that generates a random sequence of probability measures constructed\u0000using minima of increasingly accurate estimates of the influence function. We\u0000demonstrate that sFW's optimality gap sequence exhibits $O(k^{-1})$ iteration\u0000complexity almost surely and in expectation for smooth convex objectives, and\u0000$O(k^{-1/2})$ (in Frank-Wolfe gap) for smooth non-convex objectives.\u0000Furthermore, we show that an easy-to-implement fixed-step, fixed-sample version\u0000of (sFW) exhibits exponential convergence to $varepsilon$-optimality. We end\u0000with a central limit theorem on the observed objective values at the sequence\u0000of generated random measures. To further intuition, we include several\u0000illustrative examples with exact influence function calculations.","PeriodicalId":501215,"journal":{"name":"arXiv - STAT - Computation","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141531754","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The purpose of this technical note is to summarize the relationship between the marginal variance and correlation length of a Gaussian random field with Mat'ern covariance and the coefficients of the corresponding partial-differential-equation (PDE)-based precision operator.
{"title":"A note on the relationship between PDE-based precision operators and Matérn covariances","authors":"Umberto Villa, Thomas O'Leary-Roseberry","doi":"arxiv-2407.00471","DOIUrl":"https://doi.org/arxiv-2407.00471","url":null,"abstract":"The purpose of this technical note is to summarize the relationship between\u0000the marginal variance and correlation length of a Gaussian random field with\u0000Mat'ern covariance and the coefficients of the corresponding\u0000partial-differential-equation (PDE)-based precision operator.","PeriodicalId":501215,"journal":{"name":"arXiv - STAT - Computation","volume":"40 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141523198","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}