Optimal design of generalized lapped orthogonal transforms: multiobjective optimization techniques and experimental results

Abstract

We explore multiobjective optimization techniques for the design of generalized lapped orthogonal transforms (GenLOTs). The design objectives handled are the coding gain, the stopband attenuation and the DC leakage. As the optimization algorithm, the pattern search method is used and also as the multiobjective optimization method, the weighted linear combination (WLC) and minmax methods are handled. Some techniques and findings to obtain good optima by these methods are presented. The experimental results show the following. The pattern search method with the strategy for selecting the starting point and the method for assigning the frequency slot has a capability to yield good optima. The WLC and minmax methods have a capability to yield various good optima by varying the relative magnitude of each weight. There is a good possibility that the solution by the pattern search method is one of the global optima in some cases.