Intelligent cross-entropy optimizer: A novel machine learning-based meta-heuristic for global optimization

Abstract

Machine Learning (ML) features are extensively applied in various domains, notably in the context of Metaheuristic (MH) optimization methods. While MHs are known for their exploitation and exploration capabilities in navigating large and complex search spaces, they are not without their inherent weaknesses. These weaknesses include slow convergence rates and a struggle to strike an optimal balance between exploration and exploitation, as well as the challenge of effective knowledge extraction from complex data. To address these shortcomings, an AI-based global optimization technique is introduced, known as the Intelligent Cross-Entropy Optimizer (ICEO). This method draws inspiration from the concept of Cross Entropy (CE), a strategy that uses Kullback–Leibler or cross-entropy divergence as a measure of closeness between two sampling distributions, and it uses the potential of Machine Learning (ML) to facilitate the extraction of knowledge from the search data to learn and guide dynamically within complex search spaces. ICEO employs the Self-Organizing Map (SOM), to train and map the intricate, high-dimensional relationships within the search space onto a reduced lattice structure. This combination empowers ICEO to effectively address the weaknesses of traditional MH algorithms. To validate the effectiveness of ICEO, a rigorous evaluation involving well-established benchmark functions, including the CEC 2017 test suite, as well as real-world engineering problems have been conducted. A comprehensive statistical analysis, employing the Wilcoxon test, ranks ICEO against other prominent optimization approaches. The results demonstrate the superiority of ICEO in achieving the optimal balance between computational efficiency, precision, and reliability. In particular, it excels in enhancing convergence rates and exploration-exploitation balance.