Journal of Multivariate Analysis最新文献

The Laplace approximation has been proposed as a method for approximating the marginal likelihood of statistical models with latent variables. However, the approximate maximum likelihood estimators derived from the Laplace approximation are often biased for binary or temporally and/or spatially correlated data. Additionally, the corresponding Hessian matrix tends to underestimates the standard errors of these approximate maximum likelihood estimators. While higher-order approximations have been suggested, they are not applicable to complex models, such as correlated random effects models, and fail to provide consistent variance estimators. In this paper, we propose an enhanced Laplace approximation that provides the true maximum likelihood estimator and its consistent variance estimator. We study its relationship with the variational Bayes method. We also define a new restricted maximum likelihood estimator for estimating dispersion parameters and study their asymptotic properties. Enhanced Laplace approximation generally demonstrates how to obtain the true restricted maximum likelihood estimators and their variance estimators. Our numerical studies indicate that the enhanced Laplace approximation provides a satisfactory maximum likelihood estimator and restricted maximum likelihood estimator, as well as their variance estimators in the frequentist perspective. The maximum likelihood estimator and restricted maximum likelihood estimator can be also interpreted as the posterior mode and marginal posterior mode under flat priors, respectively. Furthermore, we present some comparisons with Bayesian procedures under different priors.

The unified skew-$t$ (SUT) is a flexible parametric multivariate distribution that accounts for skewness and heavy tails in the data. A few of its properties can be found scattered in the literature or in a parameterization that does not follow the original one for unified skew-normal (SUN) distributions, yet a systematic study is lacking. In this work, explicit properties of the multivariate SUT distribution are presented, such as its stochastic representations, moments, SUN-scale mixture representation, linear transformation, additivity, marginal distribution, canonical form, quadratic form, conditional distribution, change of latent dimensions, Mardia measures of multivariate skewness and kurtosis, and non-identifiability issue. These results are given in a parameterization that reduces to the original SUN distribution as a sub-model, hence facilitating the use of the SUT for applications. Several models based on the SUT distribution are provided for illustration.

In this article, we present a nonparametric method for the general two-sample problem involving functional random variables modeled as elements of a separable Hilbert space $H$. First, we present a general recipe based on linear projections to construct a measure of dissimilarity between two probability distributions on $H$. In particular, we consider a measure based on the energy statistic and present some of its nice theoretical properties. A plug-in estimator of this measure is used as the test statistic to construct a general two-sample test. Large sample distribution of this statistic is derived both under null and alternative hypotheses. However, since the quantiles of the limiting null distribution are analytically intractable, the test is calibrated using the permutation method. We prove the large sample consistency of the resulting permutation test under fairly general assumptions. We also study the efficiency of the proposed test by establishing a new local asymptotic normality result for functional random variables. Using that result, we derive the asymptotic distribution of the permuted test statistic and the asymptotic power of the permutation test under local contiguous alternatives. This establishes that the permutation test is statistically efficient in the Pitman sense. Extensive simulation studies are carried out and a real data set is analyzed to compare the performance of our proposed test with some state-of-the-art methods.

Testing the dependency between two random variables is an important inference problem in statistics since many statistical procedures rely on the assumption that the two samples are independent. To test whether two samples are independent, a so-called HSIC (Hilbert–Schmidt Independence Criterion)-based test has been proposed. Its null distribution is approximated either by permutation or a Gamma approximation. In this paper, a new HSIC-based test is proposed. Its asymptotic null and alternative distributions are established. It is shown that the proposed test is root-$n$ consistent. A three-cumulant matched chi-squared-approximation is adopted to approximate the null distribution of the test statistic. By choosing a proper reproducing kernel, the proposed test can be applied to many different types of data including multivariate, high-dimensional, and functional data. Three simulation studies and two real data applications show that in terms of level accuracy, power, and computational cost, the proposed test outperforms several existing tests for multivariate, high-dimensional, and functional data.

We propose a new family of directional dependence measures for multivariate distributions. The family of dependence measures is indexed by $\alpha \ge 1$. When $\alpha =1$, they measure the strength of dependence along different paths to the joint upper or lower orthant. For $\alpha$ large, they become tail-weighted dependence measures that put more weight in the joint upper or lower tails of the distribution. As $\alpha \to \infty$, we show the convergence of the directional dependence measures to the multivariate tail dependence function and characterize the convergence pattern with an asymptotic expansion. This expansion leads to a method to estimate the multivariate tail dependence function using weighted least square regression. We develop rank-based sample estimators for the tail-weighted dependence measures and establish their asymptotic distributions. The practical utility of the tail-weighted dependence measures in multivariate tail inference is further demonstrated through their application to a financial dataset.

We use a suitable version of the so-called ”kernel trick” to devise two-sample tests, especially focussed on high-dimensional and functional data. Our proposal entails a simplification of the practical problem of selecting an appropriate kernel function. Specifically, we apply a uniform variant of the kernel trick which involves the supremum within a class of kernel-based distances. We obtain the asymptotic distribution of the test statistic under the null and alternative hypotheses. The proofs rely on empirical processes theory, combined with the delta method and Hadamard directional differentiability techniques, and functional Karhunen–Loève-type expansions of the underlying processes. This methodology has some advantages over other standard approaches in the literature. We also give some experimental insight into the performance of our proposal compared to other kernel-based approaches (the original proposal by Borgwardt et al. (2006) and some variants based on splitting methods) as well as tests based on energy distances (Rizzo and Székely, 2017).

Online learning is an efficient approach in machine learning and statistics, which iteratively updates models upon the observation of a sequence of training examples. A representative online learning algorithm is the online gradient descent, which has found wide applications due to its low complexity and scalability to large datasets. Kernel-based learning methods have been proven to be quite successful in dealing with nonlinearity in the data and multivariate optimization. In this paper we present a class of kernel-based online gradient descent algorithm for addressing regression problems, which generates sparse estimators in an iterative way to reduce the algorithmic complexity for training streaming datasets and model selection in large-scale learning scenarios. In the setting of support vector regression (SVR), we design the sparse online learning algorithm by introducing a sequence of insensitive distance-based loss functions. We prove consistency and error bounds quantifying the generalization performance of such algorithms under mild conditions. The theoretical results demonstrate the interplay between statistical accuracy and sparsity property during learning processes. We show that the insensitive parameter plays a crucial role in providing sparsity as well as fast convergence rates. The numerical experiments also support our theoretical results.

The classical calibration procedures of computer models only concern the univariate output, which would not be satisfied in practice. Multivariate output is gradually more prevalent in a wide range of real-world applications, which motivates us to develop a new calibration procedure to extend the classical calibration methods to multivariate cases. In this work, we propose an efficient calibration procedure for multivariate output within restricted correlation. First, we construct an estimator of the discrepancy function between the true process and the computer model by the local linear approximation, then obtain an estimator of the calibration parameter by the weighted profile least squares and establish its asymptotic properties. In addition, we also develop an estimator of the calibration parameter in a special situation, whose asymptotic normality has been derived. Numerical studies including simulations and an application to composite fuselage simulation verify the efficiency of the proposed calibration procedure.

Consider the estimation of an extreme quantile region corresponding to a very small probability. Estimation of extreme quantile regions is important but difficult since extreme regions contain only a few or no observations. In this article, we propose an affine equivariant extreme quantile region estimator for heavy-tailed elliptical distributions. The estimator is constructed by extending a well-known univariate extreme quantile estimator. Consistency of the estimator is proved under estimated location and scatter. The practicality of the developed estimator is illustrated with simulations and a real data example.

In the context of large samples, a small number of individuals might spoil basic statistical indicators like the mean. It is difficult to detect automatically these atypical individuals, and an alternative strategy is using robust approaches. This paper focuses on estimating the geometric median of a random variable, which is a robust indicator of central tendency. In order to deal with large samples of data arriving sequentially, online stochastic Newton algorithms for estimating the geometric median are introduced and we give their rates of convergence. Since estimates of the median and those of the Hessian matrix can be recursively updated, we also determine confidences intervals of the median in any designated direction and perform online statistical tests.