Beyond metrics: shifting measures of research impact from citations and journal rankings to meaningful impacts on society

Abstract

Purpose

The purpose of this paper is to point out that most research published in hospitality and tourism industry journals is of no value to the visitor industries or society. It argues that a primary reason is the academic practice of evaluating the utility of its research with the inappropriate metrics of journal impact factors and citation counts. To become relevant, academic leaders must refocus evaluation of the field’s research on its meaningful impacts on society.

Design/methodology/approach

This study reviews the literature that differentiates the constructs of research quality and research impact and identifies their limitations.

Findings

The contention that either journal impact factors or citation counts adequately measure the usefulness of research is challenged. It is suggested that reliance on them has resulted in an academic self-serving “echo chamber.” The author argues the field should refocus away from the echo chamber to shift the primary evaluation of research to societal impact. Illustrations are offered from the author’s recent experiences that illustrate the fallacy of confusing citation counts with impact. The challenges of shifting to an impact assessment from the existing system are acknowledged, and a way forward is proposed.

Research limitations/implications

Continuing to embrace beguiling measures that are invested with a conventional, but false, aura of truth inevitably means the field’s scholarship will continue to be ignored.

Originality/value

An analytical critique of the prevailing metrics is undertaken, and a five-step process to shift the emphasis to societal impact is offered.