A Few Interactions Improve Distributed Nonparametric Estimation, Optimally

Abstract

Consider the problem of nonparametric estimation of an unknown $\beta $ -Hölder smooth density $p_{XY}$ at a given point, where $X$ and $Y$ are both $d$ dimensional. An infinite sequence of i.i.d. samples $(X_{i},Y_{i})$ are generated according to this distribution, and two terminals observe $(X_{i})$ and $(Y_{i})$ , respectively. They are allowed to exchange $k$ bits either in oneway or interactively in order for Bob to estimate the unknown density. We show that the minimax mean square risk is order $\left ({\frac {k}{\log k} }\right)^{-\frac {2\beta }{d+2\beta }}$ for one-way protocols and $k^{-\frac {2\beta }{d+2\beta }}$ for interactive protocols. The logarithmic improvement is nonexistent in the parametric counterparts, and therefore can be regarded as a consequence of nonparametric nature of the problem. Moreover, a few rounds of interactions achieve the interactive minimax rate: the number of rounds can grow as slowly as the super-logarithm (i.e., inverse tetration) of $k$ . The proof of the upper bound is based on a novel multi-round scheme for estimating the joint distribution of a pair of biased Bernoulli variables, and the lower bound is built on a sharp estimate of a symmetric strong data processing constant for biased Bernoulli variables.