Min Wang, Xingzhong Wang, Wei Luo, Yixue Huang, Yuanqiang Yu
{"title":"自适应模糊逻辑规则指导下加速深度强化学习","authors":"Min Wang, Xingzhong Wang, Wei Luo, Yixue Huang, Yuanqiang Yu","doi":"10.1109/phm2022-london52454.2022.00068","DOIUrl":null,"url":null,"abstract":"While Deep Reinforcement Learning (DRL) has emerged as a prospective method to many tough tasks, it remains laborious to train DRL agents with a handful of data collection and high sample efficiency. In this paper, we present an Adaptive Fuzzy Reinforcement Learning framework (AFuRL) for accelerating the learning process by incorporating adaptive fuzzy logic rules, enabling DRL agents to improve the efficiency of exploring the state space. In AFuRL, the DRL agent first leverages prior fuzzy logic rules designed especially for the actor-critic framework to learn some near-optimal policies, then further improves these policies by automatically generating adaptive fuzzy rules from state-action pairs. Ultimately, the RL algorithm is applied to refine the rough policy obtained by a fuzzy controller. We demonstrate the validity of AFuRL in both discrete and continuous control tasks, where our method surpasses DRL algorithms by a substantial margin. The experiment results show that AFuRL can find superior policies in comparison with imitation-based and some prior knowledge-based approaches.","PeriodicalId":269605,"journal":{"name":"2022 Prognostics and Health Management Conference (PHM-2022 London)","volume":"31 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2022-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Accelerating Deep Reinforcement Learning Under the Guidance of Adaptive Fuzzy Logic Rules\",\"authors\":\"Min Wang, Xingzhong Wang, Wei Luo, Yixue Huang, Yuanqiang Yu\",\"doi\":\"10.1109/phm2022-london52454.2022.00068\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"While Deep Reinforcement Learning (DRL) has emerged as a prospective method to many tough tasks, it remains laborious to train DRL agents with a handful of data collection and high sample efficiency. In this paper, we present an Adaptive Fuzzy Reinforcement Learning framework (AFuRL) for accelerating the learning process by incorporating adaptive fuzzy logic rules, enabling DRL agents to improve the efficiency of exploring the state space. In AFuRL, the DRL agent first leverages prior fuzzy logic rules designed especially for the actor-critic framework to learn some near-optimal policies, then further improves these policies by automatically generating adaptive fuzzy rules from state-action pairs. Ultimately, the RL algorithm is applied to refine the rough policy obtained by a fuzzy controller. We demonstrate the validity of AFuRL in both discrete and continuous control tasks, where our method surpasses DRL algorithms by a substantial margin. The experiment results show that AFuRL can find superior policies in comparison with imitation-based and some prior knowledge-based approaches.\",\"PeriodicalId\":269605,\"journal\":{\"name\":\"2022 Prognostics and Health Management Conference (PHM-2022 London)\",\"volume\":\"31 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2022-05-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2022 Prognostics and Health Management Conference (PHM-2022 London)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/phm2022-london52454.2022.00068\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2022 Prognostics and Health Management Conference (PHM-2022 London)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/phm2022-london52454.2022.00068","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Accelerating Deep Reinforcement Learning Under the Guidance of Adaptive Fuzzy Logic Rules
While Deep Reinforcement Learning (DRL) has emerged as a prospective method to many tough tasks, it remains laborious to train DRL agents with a handful of data collection and high sample efficiency. In this paper, we present an Adaptive Fuzzy Reinforcement Learning framework (AFuRL) for accelerating the learning process by incorporating adaptive fuzzy logic rules, enabling DRL agents to improve the efficiency of exploring the state space. In AFuRL, the DRL agent first leverages prior fuzzy logic rules designed especially for the actor-critic framework to learn some near-optimal policies, then further improves these policies by automatically generating adaptive fuzzy rules from state-action pairs. Ultimately, the RL algorithm is applied to refine the rough policy obtained by a fuzzy controller. We demonstrate the validity of AFuRL in both discrete and continuous control tasks, where our method surpasses DRL algorithms by a substantial margin. The experiment results show that AFuRL can find superior policies in comparison with imitation-based and some prior knowledge-based approaches.