Parallel Population-Based Simulated Annealing for High-Dimensional Black-Box Optimization

In this paper, we present a simple yet efficient parallel version of simulated annealing (SA) for large-scale black-box optimization within the popular population-based framework. To achieve scalability, we adopt the island model, commonly used in parallel evolutionary algorithms, to update and communicate multiple independent SA instances. For maximizing efficiency, the copy-on-write operator is used to avoid performance-expensive lock when different instances exchange solutions. For better local search ability, individual step sizes are dynamically adjusted and learned during decomposition. Furthermore, we utilize the shared memory to reduce data redundancy and support concurrent fitness evaluations for challenging problems with costly memory consumption. Experiments based on the powerful Ray distributed computing library empirically demonstrate the effectiveness and efficiency of our parallel version on a set of 2000-dimensional benchmark functions (especially each is rotated with a 2000*2000 orthogonal matrix). To the best of our knowledge, these rotated functions with a memory-expensive data matrix were not tested in all previous works which considered only much lower dimensions. For reproducibility and benchmarking, the source code is made available at https://github.com/Evolutionary-Intelligence/PPSA.