Optimizing deep reinforcement learning in data-scarce domains: a cross-domain evaluation of double DQN and dueling DQN

Abstract

The challenge of limited labeled data is a persistent concern across diverse domains, including healthcare, niche agricultural practices, astronomy and space exploration, anomaly detection, and many more. Limited data can lead to biased training, overfitting, and poor generalization in Artificial Intelligence (AI) models. In response to this ubiquitous problem, this research explores the potential of deep reinforcement learning (DRL) algorithms, specifically Double Deep Q-Network (Double DQN) and Dueling Deep Q-Network (Dueling DQN). The algorithms were trained on small training subsets generated by subsampling from the original training datasets. In this subsampling process, 10, 20, 30, and 40 instances were selected from each class to form the smaller training subsets. Subsequently, the performance of these algorithms was comprehensively assessed by evaluating them on the entire test set. We employed datasets from two different domains where this problem mainly exists to assess their performance in data-constrained scenarios. A comparative analysis was conducted against a transfer learning approach widely employed to tackle similar challenges. The comprehensive evaluation reveals compelling results. In the medical domain, Dueling DQN consistently outperformed Double DQN and transfer learning, while in the agriculture domain, Double DQN demonstrates superior performance compared to Dueling DQN and transfer learning. These findings underscore the remarkable effectiveness of DRL algorithms in addressing data scarcity across a spectrum of domains, positioning DRL as a potent tool for enhancing diverse applications with limited labeled data.