Improving Reinforcement Learning Performance through a Behavioral Psychology-Inspired Variable Reward Scheme

Abstract

Reinforcement learning (RL) algorithms employ a fixed-ratio schedule which can lead to overfitting, where the agent learns to optimize for the specific rewards it receives, rather than learning the underlying task. Further, the agent can simply repeat the same actions that have worked in the past and do not explore different actions and strategies to see what works best. This leads to generalization issue, where the agent struggles to apply what it has learned to new, unseen situations. This can be particularly problematic in complex environments where the agent needs to learn to generalize from limited data. Introducing variable reward schedules in RL inspired from behavioral psychology can be more effective than traditional reward schemes because they can mimic real-world environments where rewards are not always consistent or predictable. This can also encourage an RL agent to explore more and become more adaptable to changes in the environment. The simulation results showed that variable reward scheme has faster learning rate as compared to fixed rewards.