A reinforcement learning based mobile charging sequence scheduling algorithm for optimal sensing coverage in wireless rechargeable sensor networks

3区计算机科学 Q1 Computer Science Journal of Ambient Intelligence and Humanized Computing Pub Date : 2024-04-06 DOI:10.1007/s12652-024-04781-3

{"title":"A reinforcement learning based mobile charging sequence scheduling algorithm for optimal sensing coverage in wireless rechargeable sensor networks","authors":"","doi":"10.1007/s12652-024-04781-3","DOIUrl":null,"url":null,"abstract":"<h3>Abstract</h3> <p>Mobile charging provides a new way for energy replenishment in the Wireless Rechargeable Sensor Network (WRSN), where the Mobile Charger (MC) is employed for charging nodes sequentially via wireless energy transfer according to the mobile charging sequence scheduling result. Mobile Charging Sequence Scheduling for Optimal Sensing Coverage (MCSS-OSC) is a critical problem for providing network application performance; it aims to maximize the Quality of Sensing Coverage (QSC) of the network by optimizing the MC’s mobile charging sequence and remains a challenging problem due to its NP-completeness in nature. In this paper, we propose a novel Improved Q-learning Algorithm (IQA) for MCSS-OSC, where MC is taken as an agent to continuously learn the space of mobile charging strategies through approximate estimation and improve the charging strategy by interacting with the network environment. A novel reward function is designed according to the network sensing coverage contribution to evaluate the MC charging action at each charging time step. In addition, an efficient exploration strategy is also designed by introducing an optimal experience-strengthening mechanism to record the current optimal mobile charging sequence regularly. Extensive simulation results via Matlab2021 software show that IQA is superior to existing heuristic algorithms in network QSC, especially for large-scale networks. This paper provides an efficient solution for WRSN energy management and new ideas for performance optimization of reinforcement learning algorithms.</p>","PeriodicalId":14959,"journal":{"name":"Journal of Ambient Intelligence and Humanized Computing","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Ambient Intelligence and Humanized Computing","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s12652-024-04781-3","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"Computer Science","Score":null,"Total":0}

引用次数: 0

Abstract

Mobile charging provides a new way for energy replenishment in the Wireless Rechargeable Sensor Network (WRSN), where the Mobile Charger (MC) is employed for charging nodes sequentially via wireless energy transfer according to the mobile charging sequence scheduling result. Mobile Charging Sequence Scheduling for Optimal Sensing Coverage (MCSS-OSC) is a critical problem for providing network application performance; it aims to maximize the Quality of Sensing Coverage (QSC) of the network by optimizing the MC’s mobile charging sequence and remains a challenging problem due to its NP-completeness in nature. In this paper, we propose a novel Improved Q-learning Algorithm (IQA) for MCSS-OSC, where MC is taken as an agent to continuously learn the space of mobile charging strategies through approximate estimation and improve the charging strategy by interacting with the network environment. A novel reward function is designed according to the network sensing coverage contribution to evaluate the MC charging action at each charging time step. In addition, an efficient exploration strategy is also designed by introducing an optimal experience-strengthening mechanism to record the current optimal mobile charging sequence regularly. Extensive simulation results via Matlab2021 software show that IQA is superior to existing heuristic algorithms in network QSC, especially for large-scale networks. This paper provides an efficient solution for WRSN energy management and new ideas for performance optimization of reinforcement learning algorithms.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

基于强化学习的移动充电序列调度算法，用于优化无线充电传感器网络中的传感覆盖范围

摘要移动充电为无线可充电传感器网络（WRSN）提供了一种新的能量补充方式，根据移动充电序列调度结果，移动充电器（MC）通过无线能量传输按顺序为节点充电。优化传感覆盖的移动充电序列调度（MCSS-OSC）是提供网络应用性能的一个关键问题；它旨在通过优化 MC 的移动充电序列，最大限度地提高网络的传感覆盖质量（QSC）。在本文中，我们针对 MCSS-OSC 提出了一种新颖的改进 Q-learning 算法（IQA），将 MC 作为一个代理，通过近似估计不断学习移动充电策略空间，并通过与网络环境的交互改进充电策略。根据网络感知覆盖贡献设计了一种新的奖励函数，用于评估 MC 在每个充电时间步的充电行动。此外，还设计了一种高效的探索策略，通过引入最佳经验强化机制，定期记录当前最佳移动充电序列。通过 Matlab2021 软件进行的大量仿真结果表明，在网络 QSC 中，IQA 优于现有的启发式算法，尤其是在大规模网络中。本文为 WRSN 能量管理提供了有效的解决方案，也为强化学习算法的性能优化提供了新思路。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Journal of Ambient Intelligence and Humanized Computing COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCEC-COMPUTER SCIENCE, INFORMATION SYSTEMS

CiteScore

9.60

自引率

0.00%

发文量

854

期刊介绍： The purpose of JAIHC is to provide a high profile, leading edge forum for academics, industrial professionals, educators and policy makers involved in the field to contribute, to disseminate the most innovative researches and developments of all aspects of ambient intelligence and humanized computing, such as intelligent/smart objects, environments/spaces, and systems. The journal discusses various technical, safety, personal, social, physical, political, artistic and economic issues. The research topics covered by the journal are (but not limited to): Pervasive/Ubiquitous Computing and Applications Cognitive wireless sensor network Embedded Systems and Software Mobile Computing and Wireless Communications Next Generation Multimedia Systems Security, Privacy and Trust Service and Semantic Computing Advanced Networking Architectures Dependable, Reliable and Autonomic Computing Embedded Smart Agents Context awareness, social sensing and inference Multi modal interaction design Ergonomics and product prototyping Intelligent and self-organizing transportation networks & services Healthcare Systems Virtual Humans & Virtual Worlds Wearables sensors and actuators