Affine Modeling for the Complexity of Vector Quantizers

Abstract

We use a scalar function Θ to describe the complexity of data compression systems based on vector quantizers (VQs). This function is associated with the analog hardware implementation of a VQ, as done for example in focal-plane image compression systems. The rate and distortion of a VQ are represented by a Lagrangian cost function J. In this work we propose an affine model for the relationship between J and Θ, based on several VQ encoders performing the map R^M → {1, 2, . . . ,K}. A discrete source is obtained by partitioning images into 4×4 pixel blocks and extracting M = 4 principal components from each block. To design entropy-constrained VQs (ECVQs), we use the Generalized Lloyd Algorithm. To design simple interpolative VQs (IVQs), we consider only the simplest encoder: a linear transformation, followed by a layer of M scalar quantizers in parallel – the K cells of RM are defined by a set of thresholds {t1, . . . , tT}. The T thresholds are obtained from a non-linear unconstrained optimization method based on the Nelder-Mead algorithm.The fundamental unit of complexity Θ is "transistor": we only count the transistors that are used to implement the signal processing part of a VQ analog circuit: inner products, squares, summations, winner-takes-all, and comparators. The complexity functions for ECVQs and IVQs are as follows: ΘECVQ = 2KM + 9K + 3M + 4 and ΘIVQ = 4Mw1+2Mw2+3Mb1+Mb2+4M+3T, where Mw1 and Mw2 are the numbers of multiplications by positive and by negative weights. The numbers of positive and negative bias values are Mb1 and Mb2. Since ΘECVQ and ΘIVQ are scalar functions gathering the complexities of several different operations under the same unit, they are useful for the development of models relating rate-distortion cost to complexity.Using a training set, we designed several ECVQs and plotted all (J, Θ) points on a plane with axes log10(Θ) and log10(J) (J values from a test set). An affine model log10(Θ) = a1 log10(J) + a2 became apparent; a straightforward application of least squares yields the slope and offset coefficients. This procedure was repeated for IVQs. The error between the model and the data has a variance equal to 0.005 for ECVQs and 0.02 for IVQs. To validate the ECVQ and IVQ complexity models, we repeated the design and test procedure using new training and test sets. Then, we used the previously computed complexity models to predict the Θ of the VQs designed independently: the error between the model and the data has a variance equal to 0.01 for ECVQs and 0.02 for IVQs. This shows we are able to predict the rate-distortion performance of independently designed ECVQs and IVQs. This result serves as a starting point for studies on complexity gradients between J and Θ, and as a guideline for introducing complexity constraints in the traditional entropy-constrained Lagrangian cost.