Datasets for statistical analysis become extremely large even when stored on one single machine with some difficulty. Even when the data can be stored in one machine, the computational cost would still be intimidating. We propose a divide and conquer solution to density estimation using Bayesian mixture modelling, including the infinite mixture case. The methodology can be generalised to other application problems where a Bayesian mixture model is adopted. The proposed prior on each machine or subgroup modifies the original prior on both mixing probabilities and the rest of parameters in the distributions being mixed. The ultimate estimator is obtained by taking the average of the posterior samples corresponding to the proposed prior on each subset. Despite the tremendous reduction in time thanks to data splitting, the posterior contraction rate of the proposed estimator stays the same (up to a factor) as that using the original prior when the data is analysed as a whole. Simulation studies also justify the competency of the proposed method compared to the established WASP estimator in the finite-dimension case. In addition, one of our simulations is performed in a shape-constrained deconvolution context and reveals promising results. The application to a GWAS dataset reveals the advantage over a naive divide and conquer method that uses the original prior.
We introduce a statistical method for modelling and forecasting functional panel data represented by multiple densities. Density functions are non-negative and have a constrained integral, and thus do not constitute a linear vector space. We implement a centre log-ratio transformation to transform densities into unconstrained functions. These functions exhibit cross-sectional correlation and temporal dependence. Via a functional analysis-of-variance decomposition, we decompose the unconstrained functional panel data into a deterministic trend component and a time-varying residual component. To produce forecasts for the time-varying component, a functional time series forecasting method, based on the estimation of the long-run covariance, is implemented. By combining the forecasts of the time-varying residual component with the deterministic trend component, we obtain -step-ahead forecast curves for multiple populations. Illustrated by age- and sex-specific life-table death counts in the United States, we apply our proposed method to generate forecasts of the life-table death counts for 51 states.
Networks are mathematical structures that allow the representation of complex systems by jointly modelling the elements of the system and the relationships that exist among them. To analyse different contexts or systems, methodological tools are necessary to allow for the quantitative estimation of the differences existing between two or more networks. For this purpose, various tools have been proposed in the literature. This study is an exploratory analysis of the impacts that different methods (distances and spectral methods) have on the comparative evaluation of two networks. The analyses were conducted through a simulation study that considered three different perturbation schemes to investigate the behaviour of each method with increasing randomness in the perturbation scheme (i.e., edge removal). Results show that the distances between adjacency matrices are sensitive only to changes in the network density, while spectral methods are sensitive to changes in both the network density and the degree of the nodes.
We investigate the use of multiple linked lists for population size estimation and to estimate the relationships between covariates appearing on the lists. Over the lists, the covariates aim to measure the same concept. The relationships between the covariates are not fully known because of missing values on the covariates: some cases do not appear in some lists; some cases are on one or more of the lists but have missing covariate values on some of the lists; and some cases are not observed in any list. In earlier work, multiple system estimation has been combined with latent class analysis to give a consensus estimate where an underlying dichotomous categorical covariate is measured differently in different lists. This was applied to ethnicity covariates in New Zealand with two levels, Māori and non-Māori. In this paper, we apply this approach to ethnicity covariates with a larger number of categories, and find that it produces satisfactory results with four categories. We assess the purity of the latent classes using entropy and conditional probability measures. We also examine the evolution of annual estimates from multiple lists (where one list is the population census) over 2013–2020, finding that the estimated latent class proportions are very stable. We assess the impact of disclosure control measures on the outputs.
Fused lasso regression is a popular method for identifying homogeneous groups and sparsity patterns in regression coefficients based on either the presumed order or a more general graph structure of the covariates. However, the traditional fused lasso may yield misleading outcomes in the presence of outliers. In this paper, we propose an extension of the fused lasso, namely the robust adaptive fused lasso (RAFL), which pursues homogeneity and sparsity patterns in regression coefficients while accounting for potential outliers within the data. By using Huber's loss or Tukey's biweight loss, RAFL can resist outliers in the responses or in both the responses and the covariates. We also demonstrate that when the adaptive weights are properly chosen, the proposed RAFL achieves consistency in variable selection, consistency in grouping and asymptotic normality. Furthermore, a novel optimization algorithm, which employs the alternating direction method of multipliers, embedded with an accelerated proximal gradient algorithm, is developed to solve RAFL efficiently. Our simulation study shows that RAFL offers substantial improvements in terms of both grouping accuracy and prediction accuracy compared with the fused lasso, particularly when dealing with contaminated data. Additionally, a real analysis of cookie data demonstrates the effectiveness of RAFL.
Graphical models are important tools to characterise the conditional independence structure among a set of variables. Despite the rapid development of statistical inference for high-dimensional graphical models, existing methods typically need a stringent constraint on the sample size. In this paper, we develop a new graphical projection estimator (GPE) for statistical inference in Gaussian graphical models via partially penalised regression. The suggested inference procedure takes advantage of the strong signals, which can be identified in advance, and utilises partially penalised regression to avoid the penalisation on them when constructing the GPE. It leads to enhanced inference efficiency by removing the impacts of strong signals that contribute to the bias term. We show that the proposed GPE can enjoy asymptotic normality under a relaxed constraint on the sample size, which is of the same order as that needed for consistent estimation. The usefulness of our method is demonstrated through simulations and a prostate tumour gene expression dataset.
Misspecification of the error covariance in linear models usually leads to incorrect inference and conclusions. We consider two linear models, and , with the same design matrix but different error covariance matrices. The conditions under which every representation of the best linear unbiased estimator (BLUE) of any estimable parametric vector under remains BLUE under have been well known since C.R. Rao's paper in 1971: Unified theory of linear estimation, Sankhyā Ser. A, Vol. 33, pp. 371–394. However, there are no previously published results on retaining the weighted sum of squares of errors (SSE) for non-full-rank design or error covariance matrices, and the question of when the covariance matrix of the BLUEs is also retained has been partially explored only recently. For change in any specified error covariance matrix, we provide necessary and sufficient conditions (nasc) for both BLUEs and their covariance matrix to remain unaltered and to retain this property for all submodels. We also consider nasc for SSE to be unchanged. We decompose SSE under error covariance changes, and derive nasc under which error covariance change leaves hypothesis tests for fixed-effect deletion under normality unaltered. We also show that simultaneous retention of BLUEs and both their covariance and SSE is not possible. We outline the effects of weak and strong error covariance singularity. We provide applications (via data cloning) to maintaining data confidentiality in Official Statistics without using Confidentialised Unit Record Files (CURFs), to certain types of experimental design and to estimation of fixed parameters for linear models for single nucleotide polymorphisms (SNPs) in genetics.
We discuss a broad class of difference-based estimators of the autocovariance function in a semiparametric regression model where the signal consists of the sum of a smooth function and another stepwise function whose number of jumps and locations are unknown (change points) while the errors are stationary and -dependent. We establish that the influence of the smooth part of the signal over the bias of our estimators is negligible; this is a general result as it does not depend on the distribution of the errors. We show that the influence of the unknown smooth function is negligible also in the mean squared error (MSE) of our estimators. Although we assumed Gaussian errors to derive the latter result, our finite sample studies suggest that the class of proposed estimators still show small MSE when the errors are not Gaussian. Our simulation study also demonstrates that, when the error process is mis-specified as an AR instead of an -dependent process, our proposed method can estimate autocovariances about as well as some methods specifically designed for the AR(1) case, and sometimes even better than them. We also allow both the number of change points and the magnitude of the largest jump grow with the sample size . In this case, we provide conditions on the interplay between the growth rate of these two quantities as well as the vanishing rate of the modulus of continuity (of the signal's smooth part) that ensure consistency of our autocovariance estimators. As an application, we use our approach to provide a better understanding of the possible autocovariance structure of a time series of global averaged annual temperature anomalies. Finally, the R package dbacf complements this article.
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：