Optimal Estimation of a Subset of Integers with Application to GNSS

Abstract The problem of integer or mixed integer/real valued parameter estimation in linear models is considered. It is a well-known result that for zero-mean additive Gaussian measurement noise the integer least-squares estimator is optimal in the sense of maximizing the probability of correctly estimating the full vector of integer parameters. In applications such as global navigation satellite system ambiguity resolution, it can be beneficial to resolve only a subset of all integer parameters. We derive the estimator that leads to the highest possible success rate for a given integer subset and compare its performance to suboptimal integer mappings via numerical studies. Implementation aspects of the optimal estimator as well as subset selection criteria are discussed.