Sequential Monte Carlo (SMC) samplers form an attractive alternative to MCMC for Bayesian computation. However, their performance depends strongly on the Markov kernels used to re- juvenate particles. We discuss how to calibrate automatically (using the current particles) Hamiltonian Monte Carlo kernels within SMC. To do so, we build upon the adaptive SMC ap- proach of Fearnhead and Taylor (2013), and we also suggest alternative methods. We illustrate the advantages of using HMC kernels within an SMC sampler via an extensive numerical study.
{"title":"Adaptive Tuning Of Hamiltonian Monte Carlo Within Sequential Monte Carlo","authors":"Alexander Buchholz, N. Chopin, P. Jacob","doi":"10.1214/20-ba1222","DOIUrl":"https://doi.org/10.1214/20-ba1222","url":null,"abstract":"Sequential Monte Carlo (SMC) samplers form an attractive alternative to MCMC for Bayesian computation. However, their performance depends strongly on the Markov kernels used to re- juvenate particles. We discuss how to calibrate automatically (using the current particles) Hamiltonian Monte Carlo kernels within SMC. To do so, we build upon the adaptive SMC ap- proach of Fearnhead and Taylor (2013), and we also suggest alternative methods. We illustrate the advantages of using HMC kernels within an SMC sampler via an extensive numerical study.","PeriodicalId":8446,"journal":{"name":"arXiv: Computation","volume":"16 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81158938","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-01-22DOI: 10.1016/j.ifacol.2018.09.207
A. Wigren, Lawrence M. Murray, F. Lindsten
{"title":"Improving the particle filter in high dimensions using conjugate artificial process noise","authors":"A. Wigren, Lawrence M. Murray, F. Lindsten","doi":"10.1016/j.ifacol.2018.09.207","DOIUrl":"https://doi.org/10.1016/j.ifacol.2018.09.207","url":null,"abstract":"","PeriodicalId":8446,"journal":{"name":"arXiv: Computation","volume":"7 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81132174","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2018-01-03DOI: 10.1016/J.IFACOL.2018.09.208
J. Dahlin, A. Wills, B. Ninness
{"title":"Constructing Metropolis-Hastings proposals using damped BFGS updates","authors":"J. Dahlin, A. Wills, B. Ninness","doi":"10.1016/J.IFACOL.2018.09.208","DOIUrl":"https://doi.org/10.1016/J.IFACOL.2018.09.208","url":null,"abstract":"","PeriodicalId":8446,"journal":{"name":"arXiv: Computation","volume":"108 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2018-01-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87674389","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We wish to contribute to the discussion of "Comparing Consensus Monte Carlo Strategies for Distributed Bayesian Computation" by offering our views on the current best methods for Bayesian computation, both at big-data scale and with smaller data sets, as summarized in Table 1. This table is certainly an over-simplification of a highly complicated area of research in constant (present and likely future) flux, but we believe that constructing summaries of this type is worthwhile despite their drawbacks, if only to facilitate further discussion.
{"title":"Comment: A brief survey of the current state of play for Bayesian computation in data science at Big-Data scale","authors":"D. Draper, Alexander Terenin","doi":"10.1214/17-BJPS365B","DOIUrl":"https://doi.org/10.1214/17-BJPS365B","url":null,"abstract":"We wish to contribute to the discussion of \"Comparing Consensus Monte Carlo Strategies for Distributed Bayesian Computation\" by offering our views on the current best methods for Bayesian computation, both at big-data scale and with smaller data sets, as summarized in Table 1. This table is certainly an over-simplification of a highly complicated area of research in constant (present and likely future) flux, but we believe that constructing summaries of this type is worthwhile despite their drawbacks, if only to facilitate further discussion.","PeriodicalId":8446,"journal":{"name":"arXiv: Computation","volume":"9 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2017-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73887953","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Bayesian inference of Gibbs random fields (GRFs) is often referred to as a doubly intractable problem, since the likelihood function is intractable. The exploration of the posterior distribution of such models is typically carried out with a sophisticated Markov chain Monte Carlo (MCMC) method, the exchange algorithm (Murray et al., 2006), which requires simulations from the likelihood function at each iteration. The purpose of this paper is to consider an approach to dramatically reduce this computational overhead. To this end we introduce a novel class of algorithms which use realizations of the GRF model, simulated offline, at locations specified by a grid that spans the parameter space. This strategy speeds up dramatically the posterior inference, as illustrated on several examples. However, using the pre-computed graphs introduces a noise in the MCMC algorithm, which is no longer exact. We study the theoretical behaviour of the resulting approximate MCMC algorithm and derive convergence bounds using a recent theoretical development on approximate MCMC methods.
{"title":"Efficient MCMC for Gibbs Random Fields using pre-computation","authors":"A. Boland, N. Friel, F. Maire","doi":"10.1214/18-EJS1504","DOIUrl":"https://doi.org/10.1214/18-EJS1504","url":null,"abstract":"Bayesian inference of Gibbs random fields (GRFs) is often referred to as a doubly intractable problem, since the likelihood function is intractable. The exploration of the posterior distribution of such models is typically carried out with a sophisticated Markov chain Monte Carlo (MCMC) method, the exchange algorithm (Murray et al., 2006), which requires simulations from the likelihood function at each iteration. The purpose of this paper is to consider an approach to dramatically reduce this computational overhead. To this end we introduce a novel class of algorithms which use realizations of the GRF model, simulated offline, at locations specified by a grid that spans the parameter space. This strategy speeds up dramatically the posterior inference, as illustrated on several examples. However, using the pre-computed graphs introduces a noise in the MCMC algorithm, which is no longer exact. We study the theoretical behaviour of the resulting approximate MCMC algorithm and derive convergence bounds using a recent theoretical development on approximate MCMC methods.","PeriodicalId":8446,"journal":{"name":"arXiv: Computation","volume":"129 19","pages":""},"PeriodicalIF":0.0,"publicationDate":"2017-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91402887","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In extreme value analysis, the extreme value index plays a vital role as it determines the tail heaviness of the underlying distribution and is the primary parameter required for the estimation of other extreme events. In this paper, we review the estimation of the extreme value index when observations are subject to right random censoring and the presence of covariate information. In addition, we propose some estimators of the extreme value index, including a maximum likelihood estimator from a perturbed Pareto distribution. The existing estimators and the proposed ones are compared through a simulation study under identical conditions. The results show that the performance of the estimators depend on the percentage of censoring, the underlying distribution, the size of extreme value index and the number of top order statistics. Overall, we found the proposed estimator from the perturbed Pareto distribution to be robust to censoring, size of the extreme value index and the number of top order statistics.
{"title":"A Simulation Comparison of Estimators of Conditional Extreme Value Index under Right Random Censoring","authors":"R. Minkah, T. Wet, E. N. Nortey","doi":"10.16929/ajas/337.219","DOIUrl":"https://doi.org/10.16929/ajas/337.219","url":null,"abstract":"In extreme value analysis, the extreme value index plays a vital role as it determines the tail heaviness of the underlying distribution and is the primary parameter required for the estimation of other extreme events. In this paper, we review the estimation of the extreme value index when observations are subject to right random censoring and the presence of covariate information. In addition, we propose some estimators of the extreme value index, including a maximum likelihood estimator from a perturbed Pareto distribution. The existing estimators and the proposed ones are compared through a simulation study under identical conditions. The results show that the performance of the estimators depend on the percentage of censoring, the underlying distribution, the size of extreme value index and the number of top order statistics. Overall, we found the proposed estimator from the perturbed Pareto distribution to be robust to censoring, size of the extreme value index and the number of top order statistics.","PeriodicalId":8446,"journal":{"name":"arXiv: Computation","volume":"17 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2017-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84422266","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this expository paper we abstract and describe a simple MCMC scheme for sampling from intractable target densities. The approach has been introduced in Gonc{c}alves et al. (2017a) in the specific context of jump-diffusions, and is based on the Barker's algorithm paired with a simple Bernoulli factory type scheme, the so called 2-coin algorithm. In many settings it is an alternative to standard Metropolis-Hastings pseudo-marginal method for simulating from intractable target densities. Although Barker's is well-known to be slightly less efficient than Metropolis-Hastings, the key advantage of our approach is that it allows to implement the "marginal Barker's" instead of the extended state space pseudo-marginal Metropolis-Hastings, owing to the special form of the accept/reject probability. We shall illustrate our methodology in the context of Bayesian inference for discretely observed Wright-Fisher family of diffusions.
{"title":"Barker's algorithm for Bayesian inference with intractable likelihoods","authors":"F. Gonccalves, K. Latuszy'nski, G. Roberts","doi":"10.1214/17-BJPS374","DOIUrl":"https://doi.org/10.1214/17-BJPS374","url":null,"abstract":"In this expository paper we abstract and describe a simple MCMC scheme for sampling from intractable target densities. The approach has been introduced in Gonc{c}alves et al. (2017a) in the specific context of jump-diffusions, and is based on the Barker's algorithm paired with a simple Bernoulli factory type scheme, the so called 2-coin algorithm. In many settings it is an alternative to standard Metropolis-Hastings pseudo-marginal method for simulating from intractable target densities. Although Barker's is well-known to be slightly less efficient than Metropolis-Hastings, the key advantage of our approach is that it allows to implement the \"marginal Barker's\" instead of the extended state space pseudo-marginal Metropolis-Hastings, owing to the special form of the accept/reject probability. We shall illustrate our methodology in the context of Bayesian inference for discretely observed Wright-Fisher family of diffusions.","PeriodicalId":8446,"journal":{"name":"arXiv: Computation","volume":"11 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2017-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81993822","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2017-09-13DOI: 10.1146/annurev-statistics-031017-100232
P. Fearnhead, H. Kunsch
State-space models can be used to incorporate subject knowledge on the underlying dynamics of a time series by the introduction of a latent Markov state process. A user can specify the dynamics of this process together with how the state relates to partial and noisy observations that have been made. Inference and prediction then involve solving a challenging inverse problem: calculating the conditional distribution of quantities of interest given the observations. This article reviews Monte Carlo algorithms for solving this inverse problem, covering methods based on the particle filter and the ensemble Kalman filter. We discuss the challenges posed by models with high-dimensional states, joint estimation of parameters and the state, and inference for the history of the state process. We also point out some potential new developments that will be important for tackling cutting-edge filtering applications.
{"title":"Particle Filters and Data Assimilation","authors":"P. Fearnhead, H. Kunsch","doi":"10.1146/annurev-statistics-031017-100232","DOIUrl":"https://doi.org/10.1146/annurev-statistics-031017-100232","url":null,"abstract":"State-space models can be used to incorporate subject knowledge on the underlying dynamics of a time series by the introduction of a latent Markov state process. A user can specify the dynamics of this process together with how the state relates to partial and noisy observations that have been made. Inference and prediction then involve solving a challenging inverse problem: calculating the conditional distribution of quantities of interest given the observations. This article reviews Monte Carlo algorithms for solving this inverse problem, covering methods based on the particle filter and the ensemble Kalman filter. We discuss the challenges posed by models with high-dimensional states, joint estimation of parameters and the state, and inference for the history of the state process. We also point out some potential new developments that will be important for tackling cutting-edge filtering applications.","PeriodicalId":8446,"journal":{"name":"arXiv: Computation","volume":"44 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2017-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74166009","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Bayesian inference for models with intractable likelihood functions represents a challenging suite of problems in modern statistics. In this work we analyse the Conway-Maxwell-Poisson (COM-Poisson) distribution, a two parameter generalisation of the Poisson distribution. COM-Poisson regression modelling allows the flexibility to model dispersed count data as part of a generalised linear model (GLM) with a COM-Poisson response, where exogenous covariates control the mean and dispersion level of the response. The major difficulty with COM-Poisson regression is that the likelihood function contains multiple intractable normalising constants and is not amenable to standard inference and MCMC techniques. Recent work by Chanialidis et al. (2017) has seen the development of a sampler to draw random variates from the COM-Poisson likelihood using a rejection sampling algorithm. We provide a new rejection sampler for the COM-Poisson distribution which significantly reduces the CPU time required to perform inference for COM-Poisson regression models. A novel extension of this work shows that for any intractable likelihood function with an associated rejection sampler it is possible to construct unbiased estimators of the intractable likelihood which proves useful for model selection or for use within pseudo-marginal MCMC algorithms (Andrieu and Roberts, 2009). We demonstrate all of these methods on a real-world dataset of takeover bids.
{"title":"Bayesian inference, model selection and likelihood estimation using fast rejection sampling: the Conway-Maxwell-Poisson distribution","authors":"Alan Benson, N. Friel","doi":"10.1214/20-ba1230","DOIUrl":"https://doi.org/10.1214/20-ba1230","url":null,"abstract":"Bayesian inference for models with intractable likelihood functions represents a challenging suite of problems in modern statistics. In this work we analyse the Conway-Maxwell-Poisson (COM-Poisson) distribution, a two parameter generalisation of the Poisson distribution. COM-Poisson regression modelling allows the flexibility to model dispersed count data as part of a generalised linear model (GLM) with a COM-Poisson response, where exogenous covariates control the mean and dispersion level of the response. The major difficulty with COM-Poisson regression is that the likelihood function contains multiple intractable normalising constants and is not amenable to standard inference and MCMC techniques. Recent work by Chanialidis et al. (2017) has seen the development of a sampler to draw random variates from the COM-Poisson likelihood using a rejection sampling algorithm. We provide a new rejection sampler for the COM-Poisson distribution which significantly reduces the CPU time required to perform inference for COM-Poisson regression models. A novel extension of this work shows that for any intractable likelihood function with an associated rejection sampler it is possible to construct unbiased estimators of the intractable likelihood which proves useful for model selection or for use within pseudo-marginal MCMC algorithms (Andrieu and Roberts, 2009). We demonstrate all of these methods on a real-world dataset of takeover bids.","PeriodicalId":8446,"journal":{"name":"arXiv: Computation","volume":"125 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2017-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76157471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2017-07-25DOI: 10.1590/0101-7438.2017.037.02.0365
J. Achcar, P. Ramos, E. Martinez
In this paper, we discuss computational aspects to obtain accurate inferences for the parameters of the generalized gamma (GG) distribution. Usually, the solution of the maximum likelihood estimators (MLE) for the GG distribution have no stable behavior depending on large sample sizes and good initial values to be used in the iterative numerical algorithms. From a Bayesian approach, this problem remains, but now related to the choice of prior distributions for the parameters of this model. We presented some exploratory techniques to obtain good initial values to be used in the iterative procedures and also to elicited appropriate informative priors. Finally, our proposed methodology is also considered for data sets in the presence of censorship.
{"title":"Some Computational Aspects to Find Accurate Estimates for the Parameters of the Generalized Gamma distribution","authors":"J. Achcar, P. Ramos, E. Martinez","doi":"10.1590/0101-7438.2017.037.02.0365","DOIUrl":"https://doi.org/10.1590/0101-7438.2017.037.02.0365","url":null,"abstract":"In this paper, we discuss computational aspects to obtain accurate inferences for the parameters of the generalized gamma (GG) distribution. Usually, the solution of the maximum likelihood estimators (MLE) for the GG distribution have no stable behavior depending on large sample sizes and good initial values to be used in the iterative numerical algorithms. From a Bayesian approach, this problem remains, but now related to the choice of prior distributions for the parameters of this model. We presented some exploratory techniques to obtain good initial values to be used in the iterative procedures and also to elicited appropriate informative priors. Finally, our proposed methodology is also considered for data sets in the presence of censorship.","PeriodicalId":8446,"journal":{"name":"arXiv: Computation","volume":"75 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2017-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86369645","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：