Catch Me If You Can: Deep Meta-RL for Search-and-Rescue Using LoRa UAV Networks

Abstract

Long-range (LoRa) wireless networks have been widely proposed as efficient wireless access networks for battery-constrained Internet of Things (IoT) devices. However, applying the LoRa-based IoT network in search-and-rescue (SAR) operations will have limited coverage caused by high signal attenuation due to terrestrial blockages, especially in highly remote areas. To overcome this challenge, using unmanned aerial vehicles (UAVs) as a flying LoRa gateway to transfer messages from ground LoRa nodes to the ground rescue station can be a promising solution. In this paper, an artificial intelligence-empowered SAR operation framework using a UAV-assisted LoRa network in different unknown search environments is designed and implemented. The problem of the flying LoRa (FL) gateway control policy is modeled as a partially observable Markov decision process to move the UAV towards the LoRa transmitter carried by a lost person in the known remote search area. A deep reinforcement learning (RL)-based policy is designed to determine the adaptive FL gateway trajectory in a given search environment. Then, as a general solution, a deep meta-RL framework is used for SAR in any new and unknown environments. The proposed deep meta-RL framework integrates the information of the prior FL gateway experience in the previous SAR environments to the new environment and then rapidly adapts the UAV control policy model for SAR operation in a new and unknown environment. To analyze the performance of the proposed framework in real-world scenarios, the proposed SAR system is experimentally tested in three environments: a university campus, a wide plain, and a slotted canyon at Mongasht mountain ranges, Iran. Experimental results show that if the deep meta-RL-based control policy is applied instead of the deep RL-based one, the number of SAR time slots decreases from 141 to 50. Moreover, in the slotted canyon environment, the UAV energy consumption under the deep meta-RL policy is respectively 57% and 23% less than the deep RL and Actor-Critic RL policies.