A Comparative Study on Evolutionary Algorithms and Mathematical Programming Methods for Continuous Optimization

Evolutionary algorithms and mathematical programming methods are currently the most popular optimizers for solving continuous optimization problems. Owing to the population based search strategies, evolutionary algorithms can find a set of promising solutions without using any problem-specific information. By contrast, with the assistance of gradient and other information of the functions, mathematical programming methods can quickly converge to a single optimum. While these two types of optimizers have their own advantages and disadvantages, the performance comparison between them is rarely touched. It is known that gradient descent methods generally converge faster than evolutionary algorithms, but when can evolutionary algorithms outperform gradient descent methods? How is the scalability of them? To answer these questions, this paper first gives a review of popular evolutionary algorithms and mathematical programming methods, then conducts several experiments to compare their performance from various aspects, and finally draws some conclusions.