Yuan Liu , Jiazheng Li , Juan Zou , Zhanglu Hou , Shengxiang Yang , Jinhua Zheng
{"title":"Continuous variation operator configuration for decomposition-based evolutionary multi-objective optimization","authors":"Yuan Liu , Jiazheng Li , Juan Zou , Zhanglu Hou , Shengxiang Yang , Jinhua Zheng","doi":"10.1016/j.swevo.2024.101644","DOIUrl":null,"url":null,"abstract":"<div><p>There are various multi-objective evolutionary algorithms (MOEAs) for solving multi-objective optimization problems (MOPs), and the significant difference between them lies in the way they generate offspring, which are the so-called variation operators. Since different variation operators have their own characteristics, it is often tedious to select a suitable EA for a given MOP. Even if the optimal operator is assigned, the fixed operator and hyper-parameters make it difficult to balance exploration and exploitation during the evolutionary process. It is imperative to configure variation operators and hyper-parameters automatically during the evolutionary process, which can improve the efficiency of algorithm search. However, numerous configurations only consider operators or discretize hyper-parameters, making it difficult to achieve satisfactory results. In this paper, we formulate the operator configuration as a continuous Markov Decision Process (MDP) and use a suitable Reinforcement Learning (RL) paradigm to realize the online configuration of EAs. To simplify the deployment of MDP, we adopt a decomposition-based framework and use a one-dimensional vector with a combination of weights and objectives as state spaces. In addition, we take the selection of crossover and mutation operators and the fine-tuning of their hyper-parameters as joint action spaces. With an RL technique, we expect to achieve maximum improvement in the performance of offspring on each preference by selecting an action in a given state. We further explore the effectiveness of the proposed methodology on different characteristic MOPs. Experimental results show that our method is more competitive than other configurations and state-of-the-art EAs.</p></div>","PeriodicalId":48682,"journal":{"name":"Swarm and Evolutionary Computation","volume":null,"pages":null},"PeriodicalIF":8.2000,"publicationDate":"2024-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Swarm and Evolutionary Computation","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2210650224001822","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
There are various multi-objective evolutionary algorithms (MOEAs) for solving multi-objective optimization problems (MOPs), and the significant difference between them lies in the way they generate offspring, which are the so-called variation operators. Since different variation operators have their own characteristics, it is often tedious to select a suitable EA for a given MOP. Even if the optimal operator is assigned, the fixed operator and hyper-parameters make it difficult to balance exploration and exploitation during the evolutionary process. It is imperative to configure variation operators and hyper-parameters automatically during the evolutionary process, which can improve the efficiency of algorithm search. However, numerous configurations only consider operators or discretize hyper-parameters, making it difficult to achieve satisfactory results. In this paper, we formulate the operator configuration as a continuous Markov Decision Process (MDP) and use a suitable Reinforcement Learning (RL) paradigm to realize the online configuration of EAs. To simplify the deployment of MDP, we adopt a decomposition-based framework and use a one-dimensional vector with a combination of weights and objectives as state spaces. In addition, we take the selection of crossover and mutation operators and the fine-tuning of their hyper-parameters as joint action spaces. With an RL technique, we expect to achieve maximum improvement in the performance of offspring on each preference by selecting an action in a given state. We further explore the effectiveness of the proposed methodology on different characteristic MOPs. Experimental results show that our method is more competitive than other configurations and state-of-the-art EAs.
期刊介绍:
Swarm and Evolutionary Computation is a pioneering peer-reviewed journal focused on the latest research and advancements in nature-inspired intelligent computation using swarm and evolutionary algorithms. It covers theoretical, experimental, and practical aspects of these paradigms and their hybrids, promoting interdisciplinary research. The journal prioritizes the publication of high-quality, original articles that push the boundaries of evolutionary computation and swarm intelligence. Additionally, it welcomes survey papers on current topics and novel applications. Topics of interest include but are not limited to: Genetic Algorithms, and Genetic Programming, Evolution Strategies, and Evolutionary Programming, Differential Evolution, Artificial Immune Systems, Particle Swarms, Ant Colony, Bacterial Foraging, Artificial Bees, Fireflies Algorithm, Harmony Search, Artificial Life, Digital Organisms, Estimation of Distribution Algorithms, Stochastic Diffusion Search, Quantum Computing, Nano Computing, Membrane Computing, Human-centric Computing, Hybridization of Algorithms, Memetic Computing, Autonomic Computing, Self-organizing systems, Combinatorial, Discrete, Binary, Constrained, Multi-objective, Multi-modal, Dynamic, and Large-scale Optimization.