Machine learning may not be as good as expected : Evidence from unemployment rate forecasting

Abstract

This paper proposes a training framework by rolling k-fold cross-validation to compare forecasting performance of several quantitative methods, mainly standard time series and our pre-selected machine learning methods. Using US unemployment rate, we find that: Firstly, individual machine learning constituents may not perform as good as standard time series; secondly, among on constituent basis, SVM (support vector machine) performs the best, the deep learning (RNN-LSTM) unexpectedly performs the worst; thirdly, forecasting averaging evidence shows that the automatic machine learning (autoML, h2o.ai) performs less than our pre-selected machine learning methods, and the averaged standard time series is better than autoML. We conclude that forecasting averaging is a good way to combine diversified forecasts and a suitable combination of methods depends on the data.