Improving the Machine Learning Stock Trading System: An N-Period Volatility Labeling and Instance Selection Technique

Financial technology is crucial for the sustainable development of financial systems. Algorithmic trading, a key area in financial technology, involves automated trading based on predefined rules. However, investors cannot manually analyze all market patterns and establish rules, necessitating the development of supervised learning trading systems that can discover market patterns using machine or deep learning techniques. Many studies on supervised learning trading systems rely on up–down labeling based on price differences, which overlooks the issues of nonstationarity, complexity, and noise in stock data. Therefore, this study proposes an N-period volatility trading system that addresses the limitations of up–down labeling systems. The N-period volatility trading system measures price volatility to address uncertainty and enables the construction of a stable, long-term trading system. Additionally, an instance-selection technique is utilized to address the limitations of stock data, including noise, nonlinearity, and complexity, while effectively reducing the data size. The effectiveness of the proposed model is evaluated through trading simulations of stocks comprising the NASDAQ 100 index and compared with up–down labeling trading systems. The experimental results demonstrate that the proposed N-period volatility trading system exhibits higher stability and profitability than other trading systems.