Modeling Low-risk Actions from Multivariate Time Series Data Using Distributional Reinforcement Learning

Abstract

In recent years, investment strategies on financial markets using deep learning have attracted a significant amount of research attention. The objective of these studies is to obtain investment action that has a low risk and increases profit. On the other hand, Distributional Reinforcement Learning (DRL) expands the action-value function to a discrete distribution in reinforcement learning, which can control risk. However, DRL has not yet been used to learn investment action. In this study, we construct a low-risk investment trading model using DRL. This model is backtested on Nikkei 225 dataset and compared with Deep Q Network (DQN). We evaluate performance in terms of final asset amounts, their standard deviation, and the Sharpe ratio. The experimental results show that the proposed DRL-based method can learn low-risk actions with increasing profit, outperforming the compared method DQN.