Forecasting crude oil returns in different degrees of ambiguity: Why machine learn better?

Abstract

Numerous studies have demonstrated the strong out-of-sample predictive ability of machine learning models, particularly in variable selection and dimension reduction, on crude oil price returns. We find significant disparities in out-of-sample predictive performance between these two methods under varying degrees of ambiguity, a fuzziness measure proposed by Izhakian (2020), independent of outcomes, risks, and attitudes. Variable selection methods exhibit strong out-of-sample predictive power in low ambiguity environments, but weaker performance in high ambiguity environments, whereas dimension reduction methods show the opposite pattern. Furthermore, the optimal penalty coefficient selected by variable selection methods during in-sample model fitting is highly correlated with ambiguity, indicating that the predictive ability of variable selection stems from its ability to accurately identify the correct predictors.