Statistics and Computing最新文献

Extracting low-dimensional summary statistics from large datasets is essential for efficient (likelihood-free) inference. We characterize three different classes of summaries and demonstrate their importance for correctly analyzing dimensionality reduction algorithms. We demonstrate that minimizing the expected posterior entropy (EPE) under the prior predictive distribution of the model provides a unifying principle that subsumes many existing methods; they are shown to be equivalent to, or special or limiting cases of, minimizing the EPE. We offer a unifying framework for obtaining informative summaries and propose a practical method using conditional density estimation to learn high-fidelity summaries automatically. We evaluate this approach on diverse problems, including a challenging benchmark model with a multi-modal posterior, a population genetics model, and a dynamic network model of growing trees. The results show that EPE-minimizing summaries can lead to posterior inference that is competitive with, and in some cases superior to, dedicated likelihood-based approaches, providing a powerful and general tool for practitioners.

We propose a semi-parametric two-component model for the analysis of mixed case interval censored (MCIC) data with a cured subgroup. Such data occurs when the time to an event of interest is only known to belong to an interval obtained from a sequence of, say, k random examination time points with k representing an integer. Furthermore, there is a proportion of subjects who would never be susceptible to the event. The first component of the proposed model describes the probability of cure, and it replaces the traditional generalized linear model with a more flexible support vector machine (SVM)-based approach capable of capturing complex covariate effects. The second component of the proposed model describes the survival distribution of the uncured and is modeled using a Cox proportional hazards structure to preserve the easy interpretation of covariate effects. To the best of our knowledge, this is the first work that employs a machine learning algorithm to analyze MCIC data in the presence of a cured subgroup. To estimate the model parameters, we develop an expectation maximization algorithm. A detailed simulation study demonstrates the superiority of the proposed SVM-based model. Finally, we analyze NASA's Hypobaric Decompression Sickness Data using the proposed approach.

Supplementary information: The online version contains supplementary material available at 10.1007/s11222-025-10796-3.

We propose a novel model-based clustering approach for samples of time series. We assume as a unique commonality that two observations belong to the same group if structural changes in their behaviors happen at the same time. We resort to a latent representation of structural changes in each time series, based on random orders, to induce ties among different observations. Such an approach results in a general modeling strategy and can be combined with many time-dependent models already known in the literature. Our studies have been motivated by an epidemiological problem. Specifically, we want to provide clusters of different countries of the European Union where two countries belong to the same cluster if the spreading processes of the COVID-19 virus show structural changes at the same time.

Supplementary information: The online version contains supplementary material available at 10.1007/s11222-025-10756-x.

The mixture cure rate model (MCM) is commonly used for analyzing survival data with a cured subgroup. While the prevailing approach to modeling the probability of cure involves a generalized linear model using a known parametric link function, such as the logit link function, it has limitations in capturing the complex effects of covariates on cure probability. This paper introduces a novel MCM employing a neural network-based classifier for cure probability and an accelerated failure time structure for the survival distribution of uncured patients. An expectation maximization algorithm is developed for parameter estimation. Simulation results demonstrate the superior performance of the proposed model in capturing non-linear classification boundaries compared to logit-based and spline-based MCMs, as well as other machine learning algorithms. This enhances the accuracy and precision of cured probability estimates, improving predictive accuracy. The proposed model and estimation method are applied to survival data on leukemia cancer patients, showcasing their effectiveness.

This paper presents a nonparametric bootstrap method for estimating the proportions of inliers and outliers in robust regression models. Our approach is based on the concept of stability, providing robustness against distributional assumptions and eliminating the need for pre-specified confidence levels. Through numerical experiments, we demonstrate that this method yields more accurate and stable estimates than existing alternatives. Additionally, the generated instability paths offer a valuable graphical tool for understanding the inlier and outlier distributions within the data. The method naturally extends to generalized linear models, where we find that variance-stabilizing transformations produce residuals that are well-suited for outlier detection. Applications to two real-world datasets further illustrate the practical utility of our approach in identifying outliers.

Inference for high-dimensional hidden Markov models is challenging due to the exponential-in-dimension computational cost of calculating the likelihood. To address this issue, we introduce an innovative composite likelihood approach called "Simulation Based Composite Likelihood" (SimBa-CL). With SimBa-CL, we approximate the likelihood by the product of its marginals, which we estimate using Monte Carlo sampling. In a similar vein to approximate Bayesian computation (ABC), SimBa-CL requires multiple simulations from the model, but, in contrast to ABC, it provides a likelihood approximation that guides the optimization of the parameters. Leveraging automatic differentiation libraries, it is simple to calculate gradients and Hessians to not only speed up optimization but also to build approximate confidence sets. We present extensive empirical results which validate our theory and demonstrate its advantage over SMC, and apply SimBa-CL to real-world Aphtovirus data.

Supplementary information: The online version contains supplementary material available at 10.1007/s11222-025-10584-z.

In many modern applications, discretely-observed data may be naturally understood as a set of functions. Functional data often exhibit two confounded sources of variability: amplitude (y-axis) and phase (x-axis). The extraction of amplitude and phase, a process known as registration, is essential in exploring the underlying structure of functional data in a variety of areas, from environmental monitoring to medical imaging. Critically, such data are often gathered sequentially with new functional observations arriving over time. Despite this, existing registration procedures do not sequentially update inference based on the new data, requiring model refitting. To address these challenges, we introduce a Bayesian framework for sequential registration of functional data, which updates statistical inference as new sets of functions are assimilated. This Bayesian model-based sequential learning approach utilizes sequential Monte Carlo sampling to recursively update the alignment of observed functions while accounting for associated uncertainty. Distributed computing significantly reduces computational cost relative to refitting the model using an iterative method such as Markov chain Monte Carlo on the full data. Simulation studies and comparisons reveal that the proposed approach performs well even when the target posterior distribution has a challenging structure. We apply the proposed method to three real datasets: (1) functions of annual drought intensity near Kaweah River in California, (2) annual sea surface salinity functions near Null Island, and (3) a sequence of repeated patterns in electrocardiogram signals.

Biclustering has gained interest in gene expression data analysis due to its ability to identify groups of samples that exhibit similar behaviour in specific subsets of genes (or vice versa), in contrast to traditional clustering methods that classify samples based on all genes. Despite advances, biclustering remains a challenging problem, even with cutting-edge methodologies. This paper introduces an extension of the recently proposed Spike-and-Slab Lasso Biclustering (SSLB) algorithm, termed Outcome-Guided SSLB (OG-SSLB), aimed at enhancing the identification of biclusters in gene expression analysis. Our proposed approach integrates disease outcomes into the biclustering framework through Bayesian profile regression. By leveraging additional clinical information, OG-SSLB improves the interpretability and relevance of the resulting biclusters. Comprehensive simulations and numerical experiments demonstrate that OG-SSLB achieves superior performance, with improved accuracy in estimating the number of clusters and higher consensus scores compared to the original SSLB method. Furthermore, OG-SSLB effectively identifies meaningful patterns and associations between gene expression profiles and disease states. These promising results demonstrate the effectiveness of OG-SSLB in advancing biclustering techniques, providing a powerful tool for uncovering biologically relevant insights. The OGSSLB software can be found as an R/C++ package at https://github.com/luisvargasmieles/OGSSLB.

Individual treatment effect estimation has gained significant attention in recent data science literature. This work introduces the Double Neural Network (Double-NN) method to address this problem within the framework of extended fiducial inference (EFI). In the proposed method, deep neural networks are used to model the treatment and control effect functions, while an additional neural network is employed to estimate their parameters. The universal approximation capability of deep neural networks ensures the broad applicability of this method. Numerical results highlight the superior performance of the proposed Double-NN method compared to the conformal quantile regression (CQR) method in individual treatment effect estimation. From the perspective of statistical inference, this work advances the theory and methodology for statistical inference of large models. Specifically, it is theoretically proven that the proposed method permits the model size to increase with the sample size n at a rate of $O\left(n^{\zeta }\right)$ for some $0\le \zeta <1$ , while still maintaining proper quantification of uncertainty in the model parameters. This result marks a significant improvement compared to the range $0\le \zeta <\frac{1}{2}$ required by the classical central limit theorem. Furthermore, this work provides a rigorous framework for quantifying the uncertainty of deep neural networks under the neural scaling law, representing a substantial contribution to the statistical understanding of large-scale neural network models.

Supplementary information: The online version contains supplementary material available at 10.1007/s11222-025-10624-8.

Joint models (JMs) for longitudinal and time-to-event data are an important class of biostatistical models in health and medical research. When the study population consists of heterogeneous subgroups, standard JMs may be inadequate, leading to misleading results or loss of information. Joint latent class models (JLCMs) and their variants have been proposed to incorporate latent class structures into JMs. JLCMs are useful for identifying latent subgroups, uncovering deeper insights into relationships between the outcomes, and improving prediction performance. We consider the problem of Bayesian inference for the generic form of JLCMs, which poses significant computational challenges due to the complex nature of the posterior distribution. We propose a new Bayesian inference framework to tackle these challenges. Our approach leverages state-of-the-art Markov chain Monte Carlo techniques and parallel computing for parameter estimation and model selection regarding the number of latent classes. Through a simulation study, we demonstrate the feasibility and superiority of our proposed method over the existing approach. Additionally, we provide practical guidance on model and prior specification, which has received little attention, to facilitate the implementation of such complex models. We illustrate our method using data from the PAQUID prospective cohort study, where the outcomes of interest include a longitudinal measurement of cognitive performance and time to dementia diagnosis. Our analysis provides deeper insights into the latent class characteristics underlying the study population.

Supplementary information: The online version contains supplementary material available at 10.1007/s11222-025-10647-1.