Do altmetrics follow the crowd or does the crowd follow altmetrics?

Abstract

Changes are occurring in scholarly communication as scientific discourse and research activities spread across various social media platforms. In this paper, we study altmetrics on the article and journal levels, investigating whether the online attention received by research articles is related to scholarly impact or may be due to other factors. We define a new metric, Journal Social Impact (JSI), based on eleven data sources: CiteULike, Mendeley, F1000, blogs, Twitter, Facebook, mainstream news outlets, Google Plus, Pinterest, Reddit, and sites running Stack Exchange (Q&A). We compare JSI against diverse citation-based metrics, and find that JSI significantly correlates with a number of them. These findings indicate that online attention of scholarly articles is related to traditional journal rankings and favors journals with a longer history of scholarly impact. We also find that journal-level altmetrics have strong significant correlations among themselves, compared with the weak correlations among article-level altmetrics. Another finding is that Mendeley and Twitter have the highest usage and coverage of scholarly activities. Among individual altmetrics, we find that the readership of academic social networks have the highest correlations with citation-based metrics. Our findings deepen the overall understanding of altmetrics and can assist in validating them.