Policy Updating Methods of Q Learning for Two Player Bargaining Game

Abstract

Reinforcement learning algorithms have been used to discover the strategies in game theory. This study investigates whether Q learning, one of the classic reinforcement learning methods, is capable of training bargaining players via self-play, a training paradigm used by AlphaGo, to maximum their profit. We also compare our empirical results with the known theoretic solutions and perform an comprehensive analysis upon their differences. To accomplish these, we come up with two policy updating methods used in the training process, namely alternate update and simultaneous update, which are tailored for two players who propose offers and counter-offers in an alternating manner under a time constraint enforced by the discount factors. Our experimental results have demonstrated that the values of the discount factor actually have tangible impact on how far the bargaining outcomes deviate from the game theoretic solutions.