How reproducible and reliable is geophysical research?

Abstract

Geophysical research frequently makes use of agreed-upon methodologies, formally published software, and bespoke code to process and analyse data. The reliability and repeatability of these methods is vital in maintaining the integrity of research findings and thereby avoiding the dissemination of unreliable results. In recent years there has been increased attention on aspects of reproducibility, which includes data availability, across scientific disciplines. This review considers aspects of reproducibility of geophysical studies relating to their publication in peer reviewed journals. For 100 geophysics journals it considers the extent to which reproducibility in geophysics is the focus of published literature. For 20 geophysical journals it considers a) journal policies on the requirements for providing code, software, and data for submission; and b) the availability of data and software associated for 200 published journal articles. The findings show that: 1) between 1991 and 2021 there were 72 articles with reproducibility in the title and 417 with reliability, with an overall increase in the number of articles with reproducibility or reliability as the subject over the same period; 2) while 60% of journals have a definition of research data, only 20% of journals have a requirement for a data availability statement; and 3) despite ~86% of sampled journal articles including a data availability statement, only 54% of articles have the original data accessible via data repositories or web servers, and only 49% of articles name software used. It is suggested that despite journals and authors working towards improving the availability of data and software, frequently they are not identified, or easily accessible, therefore limiting the possibility of reproducing studies.