A Generic Flexible and Scalable Framework for Hierarchical Parallelization of Population-Based Metaheuristics

Abstract

Population-based metaheuristics -such as Evolutionary Algorithms (EAs)- are one of the most popular methods for solving highly complex and large-scale optimization problems. Nevertheless, finding an adequate solution with such approaches often requires computationally intensive fitness function evaluations especially in real-world applications. To speed up the computation, exploiting modern software techniques for parallelizing population-based metaheuristics on a cluster or a cloud is a viable approach. In the present paper, a generic, flexible and scalable framework for hierarchical hybridization of distributed population-based metaheuristics in a cluster environment is introduced. Three lightweight technologies, namely microservices, container virtualization and the publish/subscribe messaging paradigm are used to develop this framework. The combination of these technologies enables easy hybridizations of different parallelization models of population-based metaheuristics, a full decoupling between services providing basic building blocks of the algorithm and a seamless deployment in a scalable runtime environment. For evaluation purposes, the EA GLEAM (General Learning Evolutionary Algorithm and Method) is exemplarily integrated into the framework and successfully deployed in a cluster environment. Scalability and applicability of the framework are explored by hybridizing the Coarse-Grained Model with the Global Model for solving the problem of unit commitment of distributed energy resources utilizing renewable energy generation. The results show that the new proposed framework introduces an excellent performance for scaling up the optimization speed of complex unit commitment optimization problems.