Should asset managers pay for economic research? A machine learning evaluation

Abstract

This paper presents the first-ever comparison of the forecasting power of two types of narratives: articles in a major daily newspaper and regular research reports released by professional forecasters. The applied testing methodology developed in ²² and extended in this paper includes two natural language processing (NLP) techniques – the sentiment analysis and the wordscores model – that are used to convert the text corpora into the NLP indices. These indices are explanatory variables in linear regression, Granger causality test, vector autoregressive model and random forest model. The paper extends this methodology by applying Latent Dirichlet Allocation (LDA) to the newspaper corpus to filter out articles that discuss topics not relevant for economic and financial analysis. The forecasting test is conducted for two major banks in Poland – BZ WBK and mbank and for major daily newspaper Rzeczpospolita, in Polish. It is shown that mbank narratives have the best forecasting power, while BZ WBK and Rzeczpospolita trade second and third place depending on the model applied. In the vast majority of analyzed cases adding an NLP index to the model improves the forecast accuracy. The answer to the title question is – it depends. Before paying for economic research asset managers are advised to apply methods such as presented in this paper to evaluate whether sell-side research offers any forecasting value in comparison with a newspaper.