基于改进深度Q网络算法的放射性环境下移动机器人路径规划新方法

IF 2.2 3区 综合性期刊 Q2 MULTIDISCIPLINARY SCIENCES Symmetry-Basel Pub Date : 2023-11-11 DOI:10.3390/sym15112048
Zhiqiang Wu, Yebo Yin , Jie Liu, De Zhang, Jie Chen, Wei Jiang
{"title":"基于改进深度Q网络算法的放射性环境下移动机器人路径规划新方法","authors":"Zhiqiang Wu, Yebo Yin , Jie Liu, De Zhang, Jie Chen, Wei Jiang","doi":"10.3390/sym15112048","DOIUrl":null,"url":null,"abstract":"The path planning problem of nuclear environment robots refers to finding a collision-free path under the constraints of path length and an accumulated radiation dose. To solve this problem, the Improved Dueling Deep Double Q Network algorithm (ID3QN) based on asymmetric neural network structure was proposed. To address the issues of overestimation and low sample utilization in the traditional Deep Q Network (DQN) algorithm, we optimized the neural network structure and used the double network to estimate action values. We also improved the action selection mechanism, adopted a priority experience replay mechanism, and redesigned the reward function. To evaluate the efficiency of the proposed algorithm, we designed simple and complex radioactive grid environments for comparison. We compared the ID3QN algorithm with traditional algorithms and some deep reinforcement learning algorithms. The simulation results indicate that in the simple radioactive grid environment, the ID3QN algorithm outperforms traditional algorithms such as A*, GA, and ACO in terms of path length and accumulated radiation dosage. Compared to other deep reinforcement learning algorithms, including DQN and some improved DQN algorithms, the ID3QN algorithm reduced the path length by 15.6%, decreased the accumulated radiation dose by 23.5%, and converged approximately 2300 episodes faster. In the complex radioactive grid environment, the ID3QN algorithm also outperformed the A*, GA, ACO, and other deep reinforcement learning algorithms in terms of path length and an accumulated radiation dose. Furthermore, the ID3QN algorithm can plan an obstacle-free optimal path with a low radiation dose even in complex environments. These results demonstrate that the ID3QN algorithm is an effective approach for solving robot path planning problems in nuclear environments, thereby enhancing the safety and reliability of robots in such environments.","PeriodicalId":48874,"journal":{"name":"Symmetry-Basel","volume":"33 5","pages":"0"},"PeriodicalIF":2.2000,"publicationDate":"2023-11-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A Novel Path Planning Approach for Mobile Robot in Radioactive Environment Based on Improved Deep Q Network Algorithm\",\"authors\":\"Zhiqiang Wu, Yebo Yin , Jie Liu, De Zhang, Jie Chen, Wei Jiang\",\"doi\":\"10.3390/sym15112048\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The path planning problem of nuclear environment robots refers to finding a collision-free path under the constraints of path length and an accumulated radiation dose. To solve this problem, the Improved Dueling Deep Double Q Network algorithm (ID3QN) based on asymmetric neural network structure was proposed. To address the issues of overestimation and low sample utilization in the traditional Deep Q Network (DQN) algorithm, we optimized the neural network structure and used the double network to estimate action values. We also improved the action selection mechanism, adopted a priority experience replay mechanism, and redesigned the reward function. To evaluate the efficiency of the proposed algorithm, we designed simple and complex radioactive grid environments for comparison. We compared the ID3QN algorithm with traditional algorithms and some deep reinforcement learning algorithms. The simulation results indicate that in the simple radioactive grid environment, the ID3QN algorithm outperforms traditional algorithms such as A*, GA, and ACO in terms of path length and accumulated radiation dosage. Compared to other deep reinforcement learning algorithms, including DQN and some improved DQN algorithms, the ID3QN algorithm reduced the path length by 15.6%, decreased the accumulated radiation dose by 23.5%, and converged approximately 2300 episodes faster. In the complex radioactive grid environment, the ID3QN algorithm also outperformed the A*, GA, ACO, and other deep reinforcement learning algorithms in terms of path length and an accumulated radiation dose. Furthermore, the ID3QN algorithm can plan an obstacle-free optimal path with a low radiation dose even in complex environments. These results demonstrate that the ID3QN algorithm is an effective approach for solving robot path planning problems in nuclear environments, thereby enhancing the safety and reliability of robots in such environments.\",\"PeriodicalId\":48874,\"journal\":{\"name\":\"Symmetry-Basel\",\"volume\":\"33 5\",\"pages\":\"0\"},\"PeriodicalIF\":2.2000,\"publicationDate\":\"2023-11-11\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Symmetry-Basel\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.3390/sym15112048\",\"RegionNum\":3,\"RegionCategory\":\"综合性期刊\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"MULTIDISCIPLINARY SCIENCES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Symmetry-Basel","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.3390/sym15112048","RegionNum":3,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"MULTIDISCIPLINARY SCIENCES","Score":null,"Total":0}
引用次数: 0

摘要

核环境机器人的路径规划问题是指在路径长度和累积辐射剂量的约束下,寻找一条无碰撞的路径。为了解决这一问题,提出了基于非对称神经网络结构的改进Dueling Deep双Q Network算法(ID3QN)。针对传统Deep Q Network (DQN)算法中存在的预估过高和样本利用率低的问题,优化了神经网络结构,采用双网络对动作值进行预估。我们还改进了行动选择机制,采用了优先体验重放机制,并重新设计了奖励功能。为了评估该算法的效率,我们设计了简单和复杂的放射性网格环境进行比较。我们将ID3QN算法与传统算法和一些深度强化学习算法进行了比较。仿真结果表明,在简单的放射性网格环境下,ID3QN算法在路径长度和累积辐射剂量方面优于传统算法A*、GA和ACO。与其他深度强化学习算法(包括DQN和一些改进的DQN算法)相比,ID3QN算法的路径长度缩短了15.6%,累积辐射剂量减少了23.5%,收敛速度约为2300集。在复杂的放射性网格环境下,ID3QN算法在路径长度和累积辐射剂量方面也优于A*、GA、ACO等深度强化学习算法。此外,即使在复杂的环境中,ID3QN算法也可以规划出低辐射剂量的无障碍最优路径。这些结果表明,ID3QN算法是解决核环境中机器人路径规划问题的有效方法,从而提高了机器人在核环境中的安全性和可靠性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
A Novel Path Planning Approach for Mobile Robot in Radioactive Environment Based on Improved Deep Q Network Algorithm
The path planning problem of nuclear environment robots refers to finding a collision-free path under the constraints of path length and an accumulated radiation dose. To solve this problem, the Improved Dueling Deep Double Q Network algorithm (ID3QN) based on asymmetric neural network structure was proposed. To address the issues of overestimation and low sample utilization in the traditional Deep Q Network (DQN) algorithm, we optimized the neural network structure and used the double network to estimate action values. We also improved the action selection mechanism, adopted a priority experience replay mechanism, and redesigned the reward function. To evaluate the efficiency of the proposed algorithm, we designed simple and complex radioactive grid environments for comparison. We compared the ID3QN algorithm with traditional algorithms and some deep reinforcement learning algorithms. The simulation results indicate that in the simple radioactive grid environment, the ID3QN algorithm outperforms traditional algorithms such as A*, GA, and ACO in terms of path length and accumulated radiation dosage. Compared to other deep reinforcement learning algorithms, including DQN and some improved DQN algorithms, the ID3QN algorithm reduced the path length by 15.6%, decreased the accumulated radiation dose by 23.5%, and converged approximately 2300 episodes faster. In the complex radioactive grid environment, the ID3QN algorithm also outperformed the A*, GA, ACO, and other deep reinforcement learning algorithms in terms of path length and an accumulated radiation dose. Furthermore, the ID3QN algorithm can plan an obstacle-free optimal path with a low radiation dose even in complex environments. These results demonstrate that the ID3QN algorithm is an effective approach for solving robot path planning problems in nuclear environments, thereby enhancing the safety and reliability of robots in such environments.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Symmetry-Basel
Symmetry-Basel MULTIDISCIPLINARY SCIENCES-
CiteScore
5.40
自引率
11.10%
发文量
2276
审稿时长
14.88 days
期刊介绍: Symmetry (ISSN 2073-8994), an international and interdisciplinary scientific journal, publishes reviews, regular research papers and short notes. Our aim is to encourage scientists to publish their experimental and theoretical research in as much detail as possible. There is no restriction on the length of the papers. Full experimental and/or methodical details must be provided, so that results can be reproduced.
期刊最新文献
The Chromatic Entropy of Linear Supertrees and Its Application Angular-Momentum Modes in a Bosonic Condensate Trapped in the Inverse-Square Potential Development of Compact Bandpass Filter Using Symmetrical Metamaterial Structures for GPS, ISM, Wi-MAX, and WLAN Applications Entanglement and Symmetry Structure of N(= 3) Quantum Oscillators with Disparate Coupling Strengths in a Common Quantum Field Bath Nuclear Shape-Phase Transitions and the Sextic Oscillator
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1