Risk-Aware Federated Reinforcement Learning-Based Secure IoV Communications

Abstract

With the rapid growth in the number of high-mobility vehicles and booming enhanced applications with restricted latency requirements, downlink communication in Internet of Vehicles (IoV) systems has become increasingly vulnerable to active eavesdropping attacks. This paper proposes a federated learning-enabled secure communication framework for IoV against active eavesdropping, in which the roadside units (RSUs) apply reinforcement learning (RL) model to optimize their downlink transmit power levels, and the server helps update the RL models of the RSUs. First, we design a multi-agent deep RL algorithm for each RSU, which designs a punishment and a blacklist mechanism to mitigate risky explorations related to severe data leakage or communication outages. Second, this framework designs a risk-aware RL for the server, which uses a two-level hierarchical structure to choose the number of participated RSUs and the corresponding local training data size for higher optimization speed. This framework considers both the reward and risk in the selection of policies to reduce the probability of exploring the risky training policies that cause defense failure of the RSUs against active eavesdropping. Third, we analyze the convergence performance, computational complexity, and reward upper bound, which reveals how the power constraint, radio bandwidth and data size affect the secure communication performance. Simulation and experimental results validate the effectiveness of our schemes, such as the reductions of the eavesdropping rate, training latency, and the loss of local models compared to the benchmarks.