A Proximal Algorithm for Estimating the Regularized Wavelet-Based Density-Difference

Abstract

Density-Difference (DD) estimation is an important unsupervised learning procedure that proceeds many regression methods. The present work details a novel method for estimating the Difference of Densities (DoD) between two distributions. This new method directly calculates the DD, in the form of a wavelet expansion, without the need for explicitly reconstructing individual distributions. Furthermore, the method applies a regularization technique that utilizes both l2 and l1 norm penalties to robustly estimate the coefficients of the wavelet expansion. Optimizing the regularized objective is accomplished via a Proximal Gradient Descent (PGD) approach. Thus, we term our method Regularized Wavelet-based Density-Difference (RWDD) with PGD. On extensive simulated datasets, from complex multimodal to skewed distributions, our method demonstrated superior performance in comparison to other contemporary techniques.