The Scientology of Hypothesis Testing in Empirical Research: Emphasizing Economic Significance

Abstract

Over the decades, scientists across various disciplines have cautioned against the practice of over-emphasizing statistical significance at the expense of economic or practical significance. It seems that empirical researchers in the 21st century have not heeded the caution because meeting statistical significance targets continue to take precedence over the wider discovery and publication of scientifically objective findings. Statistically insignificant results are rarely reported, in part due to publication bias towards statistically significant findings. The survey finds that although large sample sizes are widely used in empirical economics and finance research, none of the surveyed papers adopted alternative methods of hypothesis testing, such as Bayesian methods. All of them explicitly or implicitly used the classical Fisher’s hypothesis testing methods. This study finds that discussions on economic significance in nearly all papers (almost 97% of papers) only wrote one sentence or two regarding the magnitude of the effect of the regressors and a declaration that the findings were economically significant. We recommend that to enhance research credibility, other methods of hypothesis testing such as Bayesian methods, should be adopted. Journal article publishers should encourage the publication of statistically insignificant empirical findings. Economic or practical significance should be emphasized and comprehensively discussed.