Deep reinforcement learning based spectrum prediction for bursty bands

Abstract

Spectrum prediction plays an important role for the secondary user (SU) to utilize the shared spectrum resources. However, currently utilized prediction methods are not well applied to spectrum with high burstiness, as parameters of prediction models cannot be adjusted properly. This paper studies the prediction problem of bursty bands. Specifically, we first collect real WiFi transmission data in 2.4GHz Industrial, Scientific, Medical (ISM) band which is considered to have bursty characteristics. Feature analysis of the data indicates that the spectrum occupancy law of the data is time-variant, which suggests that the performance of commonly used single prediction model could be restricted. Considering that the match between diverse spectrum states and multiple prediction models may essentially improve the prediction performance, we then propose a deep-reinforcement learning based multilayer perceptron (DRL-MLP) method to address this matching problem. The state space of the method is composed of feature vectors, and each of the vectors contains multi-dimensional feature values. Meanwhile, the action space consists of several multilayer perceptrons (MLPs) that are trained on the basis of multiple classified data sets. We finally conduct experiments with the collected real data and simulations with generated data to verify the performance of the proposed method. The results demonstrate that the proposed method significantly outperforms the state-of-the-art methods in terms of the prediction accuracy.