{"title":"Balance of exploration and exploitation: Non-cooperative game-driven evolutionary reinforcement learning","authors":"Jin Yu, Ya Zhang, Changyin Sun","doi":"10.1016/j.swevo.2024.101759","DOIUrl":null,"url":null,"abstract":"<div><div>In a complex and dynamic environment, it becomes difficult to solve problems with a single policy updating mode. Although evolutionary reinforcement learning partially addresses this issue, it fails to consider the real-time status of the agent, resulting in inflexible policy updating modes that can negatively affect algorithm performance. To address this issue, we propose an evolutionary reinforcement learning based on non-cooperative games that leverages the benefits of various algorithms in cooperative and non-cooperative settings. Firstly, a competition framework is established between the evolutionary algorithm and reinforcement learning using a non-cooperative game. The policy updating mode is dynamically selected based on the game’s outcome, ensuring diverse algorithms through differentiated population evolution guided by Nash equilibrium. Secondly, the evolutionary algorithm collaborates with the non-cooperative game to establish a cooperative framework that balances exploration and convergence. Since the exploration of the evolutionary algorithm does not rely on the environment, it is used to guide the non-cooperative game, which helps the algorithm successfully overcome the local optimum. This synergy significantly enhances algorithm performance. Experimental results demonstrate that the proposed algorithm outperforms individual algorithms.</div></div>","PeriodicalId":48682,"journal":{"name":"Swarm and Evolutionary Computation","volume":"91 ","pages":"Article 101759"},"PeriodicalIF":8.2000,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Swarm and Evolutionary Computation","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2210650224002979","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
In a complex and dynamic environment, it becomes difficult to solve problems with a single policy updating mode. Although evolutionary reinforcement learning partially addresses this issue, it fails to consider the real-time status of the agent, resulting in inflexible policy updating modes that can negatively affect algorithm performance. To address this issue, we propose an evolutionary reinforcement learning based on non-cooperative games that leverages the benefits of various algorithms in cooperative and non-cooperative settings. Firstly, a competition framework is established between the evolutionary algorithm and reinforcement learning using a non-cooperative game. The policy updating mode is dynamically selected based on the game’s outcome, ensuring diverse algorithms through differentiated population evolution guided by Nash equilibrium. Secondly, the evolutionary algorithm collaborates with the non-cooperative game to establish a cooperative framework that balances exploration and convergence. Since the exploration of the evolutionary algorithm does not rely on the environment, it is used to guide the non-cooperative game, which helps the algorithm successfully overcome the local optimum. This synergy significantly enhances algorithm performance. Experimental results demonstrate that the proposed algorithm outperforms individual algorithms.
期刊介绍:
Swarm and Evolutionary Computation is a pioneering peer-reviewed journal focused on the latest research and advancements in nature-inspired intelligent computation using swarm and evolutionary algorithms. It covers theoretical, experimental, and practical aspects of these paradigms and their hybrids, promoting interdisciplinary research. The journal prioritizes the publication of high-quality, original articles that push the boundaries of evolutionary computation and swarm intelligence. Additionally, it welcomes survey papers on current topics and novel applications. Topics of interest include but are not limited to: Genetic Algorithms, and Genetic Programming, Evolution Strategies, and Evolutionary Programming, Differential Evolution, Artificial Immune Systems, Particle Swarms, Ant Colony, Bacterial Foraging, Artificial Bees, Fireflies Algorithm, Harmony Search, Artificial Life, Digital Organisms, Estimation of Distribution Algorithms, Stochastic Diffusion Search, Quantum Computing, Nano Computing, Membrane Computing, Human-centric Computing, Hybridization of Algorithms, Memetic Computing, Autonomic Computing, Self-organizing systems, Combinatorial, Discrete, Binary, Constrained, Multi-objective, Multi-modal, Dynamic, and Large-scale Optimization.