Bibliometric Analyses: Do They Contribute to Knowledge?

Abstract

As the editors-in-chief of two prominent nursing journals, one clinical and one educational, we are aware of a relatively new trend: the submission to both our journals of manuscripts reporting studies, which are based on bibliometric analysis. Such study uses the science of bibliometrics and the emerging software packages that enable rapid analysis and visual representation of the state of publication in each field of study.

Bibliometrics is the study of academic publication in terms of number of articles published, where they are published and by whom, and the extent of mutual citation. Where they are published may be broken down by region, country and institution. Such information is vital for the academic publishing industry as it helps in the identification of trends, competition and productivity. However, whether they make a useful contribution to any particular field of academic study is an issue we wish to explore.

Often receiving several such manuscripts weekly, we note that there is a trend in the sources of such studies. We do not wish to specify here but the sources tend to be countries where the pressure to publish is high and where research students are expected to publish to qualify for their degrees. That and availability and ease of use of bibliometrics software packages provides a heady cocktail of necessity and opportunity.

The initial novelty and the ease of understanding and visual attractiveness of manuscripts using bibliometric analyses was appealing to editors; this was especially the case if the outcome of any analysis showed their journal in a good light. We consider that the honeymoon period is over, and we must review the value of bibliometric studies in nursing and suggest some parameters to help potential authors.

Most such studies are usually titled along the lines of ‘A bibliometric analysis of the published literature in…’. One of the first problems with the bibliometric approach to publication is that such studies are atheoretical and, therefore, not guided by a research question. Bibliometrics alone is the point, and this suggests, like other commonly submitted designs such as ‘mediating variables’ studies that here we have another prime example of a research method in search of a question. Another problem, related to the above, is that—as attractive and easy to understand as the visual representation of the parameters in the studies are—they are, essentially, purely descriptive offering little insight beyond the data that are presented.

Nevertheless, we would not wish to consign bibliometric studies in our field to the dustbin of history. Without wishing to create a new wave of trends in submissions, bibliometric analysis may well be useful if the right research questions are posed. For example, questions around the contribution of nursing literature related to particular fields of nursing to the general medical or scientific and social scientific literature could be addressed. Longitudinal analyses applied to such questions would reveal whether this was increasing and what the geographical distribution of such contributions is. Likewise, issues around multi- and interdisciplinarity could be addressed and studied over time.

Bibliometric analysis may well prove a useful adjunct to systematic review studies, adding a dimension over and above the efficacy and effectiveness of interventions. Insights provided by bibliometric analysis could produce useful graphics to show which regions, countries or institutions were most productive and, for example, where the best-designed studies were carried out and evidence of collaboration between researchers, between institutions and across borders.

Editors-in-chief are generally and increasingly inundated with submissions. Some of these submissions, by their titles alone, stand out and catch the eye of the editor-in-chief due to their design, novelty or currency. Other titles stand out for the opposite reasons and, unfortunately for these two editors at least, studies titled ‘A bibliometric analysis of…’ are a prime example. A cursory inspection of the abstract will soon tell us whether it is worth proceeding with such a study or not; most often, it is not.

Therefore, if you have it in mind to conduct a bibliometric study, we urge you to ask yourself why you are doing it. If you are considering conducting a bibliometric study as a means, simply, of securing yourself a publication, we urge you to think again.