Bit allocation in sub-linear time and the multiple-choice knapsack problem

Abstract

We show that the problem of optimal bit allocation among a set of independent discrete quantizers given a budget constraint is equivalent to the multiple choice knapsack problem (MCKP). This result has three implications: first, it provides a trivial proof that the problem of optimal bit allocation is NP-hard and that its related decision problem is NP-complete; second, it unifies research into solving these problems that has to date been done independently in the data compression community and the operations research community; third, many practical algorithms for approximating the optimal solution to MCKP can be used for bit allocation. We implement the GBFOS, partition-search, and Dudzinski-Walukiewicz algorithms and compare their running times for a variety of problem sizes.