Channel probing for opportunistic access with multi-channel sensing

We consider an opportunistic communication system consisting of multiple independent channels with time-varying states. We formulate the problem of optimal sequential channel selection as a restless multi-armed bandit process, for which a powerful policy-Whittle's index policy-can be implemented based on the indexability of the system. We obtain Whittle's index in closed-form under the average reward criterion, which leads to the direct implementation of Whittle's index policy. To evaluate the performance of Whittle's index policy, we provide simple algorithms to calculate an upper bound of the optimal performance. The tightness of the upper bound and the near-optimal performance of Whittle's index policy are illustrated with simulation examples. When channels are stochastically identical, we show that Whittle's index policy is equivalent to the myopic policy, which has a simple and robust structure. Based on this structure, we establish the approximation factors of the performance of Whittle's index policy. Furthermore, we show that Whittle's index policy is optimal under certain conditions.