Pub Date : 1990-07-01DOI: 10.1111/J.2517-6161.1990.TB01798.X
P. Green
SUMMARY The EM algorithm is a popular approach to maximum likelihood estimation but has not been much used for penalized likelihood or maximum a posteriori estimation. This paper discusses properties of the EM algorithm in such contexts, concentrating on rates of conver- gence, and presents an alternative that is usually more practical and converges at least as quickly. The EM algorithm is a general approach to maximum likelihood estimation, rather than a specific algorithm. Dempster et al. (1977) discussed the method and derived basic properties, demonstrating that a variety of procedures previously developed rather informally could be unified. The common strand to problems where the approach is applicable is a notion of 'incomplete data'; this includes the conventional sense of 'missing data' but is much broader than that. The EM algorithm demon- strates its strength in situations where some hypothetical experiment yields data from which estimation is particularly convenient and economical: the 'incomplete' data actually at hand are regarded as observable functions of these 'complete' data. The resulting algorithms, while usually slow to converge, are often extremely simple and remain practical in large problems where no other approaches may be feasible. Dempster et al. (1977) briefly refer to the use of the same approach to the problem of finding the posterior mode (maximum a posteriori estimate) in a Bayesian estima-
{"title":"On Use of the EM Algorithm for Penalized Likelihood Estimation","authors":"P. Green","doi":"10.1111/J.2517-6161.1990.TB01798.X","DOIUrl":"https://doi.org/10.1111/J.2517-6161.1990.TB01798.X","url":null,"abstract":"SUMMARY The EM algorithm is a popular approach to maximum likelihood estimation but has not been much used for penalized likelihood or maximum a posteriori estimation. This paper discusses properties of the EM algorithm in such contexts, concentrating on rates of conver- gence, and presents an alternative that is usually more practical and converges at least as quickly. The EM algorithm is a general approach to maximum likelihood estimation, rather than a specific algorithm. Dempster et al. (1977) discussed the method and derived basic properties, demonstrating that a variety of procedures previously developed rather informally could be unified. The common strand to problems where the approach is applicable is a notion of 'incomplete data'; this includes the conventional sense of 'missing data' but is much broader than that. The EM algorithm demon- strates its strength in situations where some hypothetical experiment yields data from which estimation is particularly convenient and economical: the 'incomplete' data actually at hand are regarded as observable functions of these 'complete' data. The resulting algorithms, while usually slow to converge, are often extremely simple and remain practical in large problems where no other approaches may be feasible. Dempster et al. (1977) briefly refer to the use of the same approach to the problem of finding the posterior mode (maximum a posteriori estimate) in a Bayesian estima-","PeriodicalId":17425,"journal":{"name":"Journal of the royal statistical society series b-methodological","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1990-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87336707","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1990-07-01DOI: 10.1111/J.2517-6161.1990.TB01796.X
A. Davison, Richard L. Smith
We discuss the analysis of the extremes of data by modelling the sizes and occurrence of exceedances over high thresholds. The natural distribution for such exceedances, the generalized Pareto distribution, is described and its properties elucidated. Estimation and model-checking procedures for univariate and regression data are developed, and the influence of and information contained in the most extreme observations in a sample are studied. Models for seasonality and serial dependence in the point process of exceedances are described. Sets of data on river flows and wave heights are discussed, and an application to the siting of nuclear installations is described
{"title":"Models for exceedances over high thresholds","authors":"A. Davison, Richard L. Smith","doi":"10.1111/J.2517-6161.1990.TB01796.X","DOIUrl":"https://doi.org/10.1111/J.2517-6161.1990.TB01796.X","url":null,"abstract":"We discuss the analysis of the extremes of data by modelling the sizes and occurrence of exceedances over high thresholds. The natural distribution for such exceedances, the generalized Pareto distribution, is described and its properties elucidated. Estimation and model-checking procedures for univariate and regression data are developed, and the influence of and information contained in the most extreme observations in a sample are studied. Models for seasonality and serial dependence in the point process of exceedances are described. Sets of data on river flows and wave heights are discussed, and an application to the siting of nuclear installations is described","PeriodicalId":17425,"journal":{"name":"Journal of the royal statistical society series b-methodological","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1990-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82624684","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1987-07-01DOI: 10.1111/J.2517-6161.1987.TB01695.X
C. S. Wallace, P. Freeman
SUMMARY The systematic variation within a set of data, as represented by a usual statistical model, may be used to encode the data in a more compact form than would be possible if they were considered to be purely random. The encoded form has two parts. The first states the inferred estimates of the unknown parameters in the model, the second states the data using an optimal code based on the data probability distribution implied by those parameter estimates. Choosing the model and the estimates that give the most compact coding leads to an interesting general inference procedure. In its strict form it has great generality and several nice properties but is computationally infeasible. An approximate form is developed and its relation to other methods is explored.
{"title":"Estimation and Inference by Compact Coding","authors":"C. S. Wallace, P. Freeman","doi":"10.1111/J.2517-6161.1987.TB01695.X","DOIUrl":"https://doi.org/10.1111/J.2517-6161.1987.TB01695.X","url":null,"abstract":"SUMMARY The systematic variation within a set of data, as represented by a usual statistical model, may be used to encode the data in a more compact form than would be possible if they were considered to be purely random. The encoded form has two parts. The first states the inferred estimates of the unknown parameters in the model, the second states the data using an optimal code based on the data probability distribution implied by those parameter estimates. Choosing the model and the estimates that give the most compact coding leads to an interesting general inference procedure. In its strict form it has great generality and several nice properties but is computationally infeasible. An approximate form is developed and its relation to other methods is explored.","PeriodicalId":17425,"journal":{"name":"Journal of the royal statistical society series b-methodological","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1987-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82274842","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1984-07-01DOI: 10.1111/J.2517-6161.1984.TB01321.X
O. Barndorff-Nielsen, D. Cox
For rather general parametric models, a simple connection is established between the Bartlett adjustment factor of the log-likelihood ratio statistic and the normalizing constant c of the formula c I I 1?2L for the conditional distribution of a maximum likelihood estimator as applied to the full model and the model of the hypothesis tested. This leads to a relatively simple demonstration that division of the likelihood ratio statistic by a suitable constant or estimated factor improves the chi-squared approximation to its distribution. Various expressions for these quantities are discussed. In particular, for the case of a one-dimensional parameter an approximation to the constants involved is derived, which does not require integration over the sample space.
对于相当一般的参数模型，对数似然比统计量的Bartlett调整因子与公式c I I 1?的规范化常数c之间建立了简单的联系。2L为条件分布的极大似然估计量，应用于完整模型和模型的假设检验。这导致了一个相对简单的演示，即用合适的常数或估计因子来划分似然比统计量可以改善其分布的卡方近似。讨论了这些量的各种表达式。特别地，对于一维参数的情况，导出了所涉及的常数的近似值，它不需要在样本空间上积分。
{"title":"Bartlett Adjustments to the Likelihood Ratio Statistic and the Distribution of the Maximum Likelihood Estimator","authors":"O. Barndorff-Nielsen, D. Cox","doi":"10.1111/J.2517-6161.1984.TB01321.X","DOIUrl":"https://doi.org/10.1111/J.2517-6161.1984.TB01321.X","url":null,"abstract":"For rather general parametric models, a simple connection is established between the Bartlett adjustment factor of the log-likelihood ratio statistic and the normalizing constant c of the formula c I I 1?2L for the conditional distribution of a maximum likelihood estimator as applied to the full model and the model of the hypothesis tested. This leads to a relatively simple demonstration that division of the likelihood ratio statistic by a suitable constant or estimated factor improves the chi-squared approximation to its distribution. Various expressions for these quantities are discussed. In particular, for the case of a one-dimensional parameter an approximation to the constants involved is derived, which does not require integration over the sample space.","PeriodicalId":17425,"journal":{"name":"Journal of the royal statistical society series b-methodological","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1984-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91431761","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1984-07-01DOI: 10.1111/J.2517-6161.1984.TB01319.X
P. Hall, D. Titterington
SUMMARY By constructing a sequence of multinomial approximations and related maximum likelihood estimators, we derive a Cramer-Rao lower bound for nonparametric estimators of the mixture proportions and thereby characterize asymptotically optimal estimators. For the case of the sampling model M2 of Hosmer (1973) it is shown that the sequence of maximum likelihood estimators, which can be obtained explicitly, is asymptotically optimal in this sense. The results hold true even when the multinomial approximations involve cells chosen adaptively, from the data, in a wellspecified way.
{"title":"Efficient Nonparametric Estimation of Mixture Proportions","authors":"P. Hall, D. Titterington","doi":"10.1111/J.2517-6161.1984.TB01319.X","DOIUrl":"https://doi.org/10.1111/J.2517-6161.1984.TB01319.X","url":null,"abstract":"SUMMARY By constructing a sequence of multinomial approximations and related maximum likelihood estimators, we derive a Cramer-Rao lower bound for nonparametric estimators of the mixture proportions and thereby characterize asymptotically optimal estimators. For the case of the sampling model M2 of Hosmer (1973) it is shown that the sequence of maximum likelihood estimators, which can be obtained explicitly, is asymptotically optimal in this sense. The results hold true even when the multinomial approximations involve cells chosen adaptively, from the data, in a wellspecified way.","PeriodicalId":17425,"journal":{"name":"Journal of the royal statistical society series b-methodological","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1984-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79284672","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1984-07-01DOI: 10.1111/J.2517-6161.1984.TB01311.X
G. Eagleson
SUMMARY A Poisson limit theorem is derived for the number of "large" values observed among comparisons of independent, but not necessarily identically distributed random variables. The comparisons made need not be the same and may depend on the two variables being compared. An application to the assessment of large numbers of correlation coefficients is given.
{"title":"Poisson Convergence for Dissociated Statistics","authors":"G. Eagleson","doi":"10.1111/J.2517-6161.1984.TB01311.X","DOIUrl":"https://doi.org/10.1111/J.2517-6161.1984.TB01311.X","url":null,"abstract":"SUMMARY A Poisson limit theorem is derived for the number of \"large\" values observed among comparisons of independent, but not necessarily identically distributed random variables. The comparisons made need not be the same and may depend on the two variables being compared. An application to the assessment of large numbers of correlation coefficients is given.","PeriodicalId":17425,"journal":{"name":"Journal of the royal statistical society series b-methodological","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1984-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86348326","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1984-07-01DOI: 10.1111/J.2517-6161.1984.TB01318.X
N. Cressie, Timothy R. C. Read
{"title":"Multinomial goodness-of-fit tests","authors":"N. Cressie, Timothy R. C. Read","doi":"10.1111/J.2517-6161.1984.TB01318.X","DOIUrl":"https://doi.org/10.1111/J.2517-6161.1984.TB01318.X","url":null,"abstract":"","PeriodicalId":17425,"journal":{"name":"Journal of the royal statistical society series b-methodological","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1984-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88459277","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1984-07-01DOI: 10.1111/J.2517-6161.1984.TB01308.X
Mark H. A. Davis
A general class of non-diffusion stochastic models is introduced with a view to providing a framework for studying optimization problems arising in queueing systems, inventory theory, resource allocation and other areas. The corresponding stochastic processes are Markov processes consisting of a mixture of deterministic motion and random jumps. Stochastic calculus for these processes is developed and a complete characterization of the extended generator is given; this is the main technical result of the paper. The relevance of the extended generator concept in applied problems is discussed and some recent results on optimal control of piecewise-deterministic processes are described.
{"title":"Piecewise‐Deterministic Markov Processes: A General Class of Non‐Diffusion Stochastic Models","authors":"Mark H. A. Davis","doi":"10.1111/J.2517-6161.1984.TB01308.X","DOIUrl":"https://doi.org/10.1111/J.2517-6161.1984.TB01308.X","url":null,"abstract":"A general class of non-diffusion stochastic models is introduced with a view to providing a framework for studying optimization problems arising in queueing systems, inventory theory, resource allocation and other areas. The corresponding stochastic processes are Markov processes consisting of a mixture of deterministic motion and random jumps. Stochastic calculus for these processes is developed and a complete characterization of the extended generator is given; this is the main technical result of the paper. The relevance of the extended generator concept in applied problems is discussed and some recent results on optimal control of piecewise-deterministic processes are described.","PeriodicalId":17425,"journal":{"name":"Journal of the royal statistical society series b-methodological","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1984-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88729229","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1984-07-01DOI: 10.1111/J.2517-6161.1984.TB01316.X
S. Bar-Lev
{"title":"Asymptotic Behaviour of Conditional Maximum Likelihood Estimators in a Certain Exponential Model","authors":"S. Bar-Lev","doi":"10.1111/J.2517-6161.1984.TB01316.X","DOIUrl":"https://doi.org/10.1111/J.2517-6161.1984.TB01316.X","url":null,"abstract":"","PeriodicalId":17425,"journal":{"name":"Journal of the royal statistical society series b-methodological","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1984-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91303972","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1984-07-01DOI: 10.1111/J.2517-6161.1984.TB01317.X
M. West
SUMMARY Bayesian inference in regression models is considered using heavy-tailed error distri- butions to accommodate outliers. The particular class of distributions that can be con- structed as scale mixtures of normal distributions are examined and use is made of them as both error models and prior distributions in Bayesian linear modelling, includ- ing simple regression and more complex hierarchical models with structured priors depending on unknown hyperprior parameters. The modelling of outliers in nominally normal linear regression models using alternative error distributions which are heavy-tailed relative to the normal provides an automatic means of both detecting and accommodating possibly aberrant observations. Such realistic models do, however, often lead to analytically intractable analyses with complex posterior distributions in several dimensions that are difficult to summarize and understand. In this paper we consider a special yet rather wide class of heavy-tailed, unimodal and symmetric error distributions for which the analyses, though apparently intractable, can be examined in some depth by exploiting certain properties of the assumed error form. The distributions concerned are those that can be con- structed as scale mixtures of normal distributions. In his paper concerning location parameters, de Finetti (1961) discusses such distributions and suggests the hypothetical interpretation that "each observation is taken using an instrument with normal error, but each time chosen at random from a collection of instruments of different precisions, the distribution of the
{"title":"Outlier Models and Prior Distributions in Bayesian Linear Regression","authors":"M. West","doi":"10.1111/J.2517-6161.1984.TB01317.X","DOIUrl":"https://doi.org/10.1111/J.2517-6161.1984.TB01317.X","url":null,"abstract":"SUMMARY Bayesian inference in regression models is considered using heavy-tailed error distri- butions to accommodate outliers. The particular class of distributions that can be con- structed as scale mixtures of normal distributions are examined and use is made of them as both error models and prior distributions in Bayesian linear modelling, includ- ing simple regression and more complex hierarchical models with structured priors depending on unknown hyperprior parameters. The modelling of outliers in nominally normal linear regression models using alternative error distributions which are heavy-tailed relative to the normal provides an automatic means of both detecting and accommodating possibly aberrant observations. Such realistic models do, however, often lead to analytically intractable analyses with complex posterior distributions in several dimensions that are difficult to summarize and understand. In this paper we consider a special yet rather wide class of heavy-tailed, unimodal and symmetric error distributions for which the analyses, though apparently intractable, can be examined in some depth by exploiting certain properties of the assumed error form. The distributions concerned are those that can be con- structed as scale mixtures of normal distributions. In his paper concerning location parameters, de Finetti (1961) discusses such distributions and suggests the hypothetical interpretation that \"each observation is taken using an instrument with normal error, but each time chosen at random from a collection of instruments of different precisions, the distribution of the","PeriodicalId":17425,"journal":{"name":"Journal of the royal statistical society series b-methodological","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"1984-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84397291","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：