An Improvement for Value-Based Reinforcement Learning Method Through Increasing Discount Factor Substitution

Abstract

Discount factor is typically considered as a constant value in conventional Reinforcement Learning (RL) methods, and the exponential inhibition is used to evaluate the future rewards that can guarantee the theoretical convergence of Bellman Equation. However, exponential inhibition mode greatly underestimates future rewards, which is obviously unreasonable. Future rewards, especially those that are closer to the completion of the task, should be given greater importance. In this paper, we review the rationale of discount factor and propose an increasing discount factor to reduce the underestimation effect of exponential inhibition on future rewards. We test two value-based reinforcement learning methods in three scenarios to verify our method. The experimental results show that value-based reinforcement learning with increasing discount factor is more efficient than it with fixed discount factor under certain circumstances.