What gives a stroke publication impact? Assessing traditional and alternative metrics of scientific impact for papers published in the journal Stroke

Abstract

Background: The ‘impact’ of a scientific paper is a measure of influence in its field. In recent years, traditional, citation-based measures of impact have been complemented by Altmetrics, which quantify social media footprint. As authors and research institutions seek to increase their visibility both within and beyond the academic community, it is important to identify and compare the determinants of traditional and alternative metrics. We explored this using Stroke – a leading journal in its field. Methods: We described the impact of original research papers published in Stroke (2015-2016) using citation count and Altmetric Attention Score (Altmetrics). Using these two metrics as our outcomes, we assessed univariable and multivariable associations with 21 plausibly relevant publication features. We set the significance threshold at p<0.01. Results: Across 911 papers published in Stroke, there was an average citation count of 21.60 (±17.40) and Altmetric score of 17.99 (±47.37). The two impact measures were weakly correlated (r=0.15, p<0.001). Citations were independently associated with five publication features at a significance level of p<0.01: Time Since Publication (beta=0.87), Number of Authors (beta=0.22), Publication Type (beta=6.76), Number of Previous Publications (beta=0.01) and Editorial (beta=9.45). For Altmetrics, we observed a trend for independent associations with: Time Since Publication (beta=-0.25, p=0.02), Number of References (beta=0.32, p=0.02) and Country of Affiliation (beta=8.59, p=0.01). Our models explained 21% and 3% of variance in citations and Altmetrics, respectively. Conclusion: Papers published in Stroke have impact. Certain aspects of content and format may contribute to impact, but these differ for traditional measures and Altmetrics, and explain only a very modest proportion of variance in the latter. Citation counts and Altmetrics seem to represent different constructs and, therefore, should be used in conjunction to allow a more comprehensive assessment of publication impact.