Multiagent Multi-Armed Bandit Techniques for Millimeter Wave Concurrent Beamforming

This paper leverages multiagent multi-armed bandit (MA-MAB) schemes to handle efficiently the millimeter wave (mmWave) concurrent transmission problem. MA-MAB selects the beams with the maximum long-term reward, mostly data rate, from the concurrent links. The mmWave access points (APs) are the agents of the bandit game, while the arms are the existing beam directions. Towards that, MA-KLUCB and MA-EXP3 are proposed and tested via applying them within each AP in a selfish concurrent beamforming scenario. It turns out that the proposed algorithms provide near-optimal performances as compared to the exhaustive search of all concurrent beams’ combinations. Also, they demonstrate reasonable fast convergence rates too.