Early Experimental Psychology: How did Replication Work Before P-Hacking?

Abstract

For many researchers, replication is still the “gold standard” that is crucial for verifying scientific findings (see, for example, Frank & Saxe, 2012; Iso-Ahola, 2020; Witte & Zenker, 2017). Indeed, Crandall and Sherman (2016) declared that: “[t]here is no controversy over the need for replication; virtually all scientists and philosophers of science endorse the notion that replication of one sort or another is absolutely essential” (p. 94). In recent decades, this has led to widespread concern because few experimental findings are actually being confirmed in this way (see, for example, Pashler & Wagenmakers, 2012; Reproducibility Project: Psychology; Wiggins & Chrisopherson, 2019). Before it is possible to plan how to remedy this situation, the reasons for the lack of replications must be identified. “Questionable research practices” such as p-hacking or post-hoc hypothesizing, the “file-drawer problem,” are often cited as contributing to the problem (Romero, 2019; Wiggins & Chrisopherson, 2019). These research practices are firmly embeddedwithin a scientific culture that is characterized by a highly competitive academic environment and a reward system that dissuades rather than encouraging replication (Crandall & Sherman, 2016; Romero, 2019). This setting fosters personal ambition, urging researchers to come up with innovative and ambitious projects continually and to publish as many papers as possible. Meanwhile, most journals only publish reports of original research offering statistically significant results, which has led to a “publication bias” (Romero, 2017). Replicability problems, as Pashler and Wagenmakers (2012) stated, “reflect deep-seated human biases and well entrenched incentives that shape the behavior of individuals and institutions” (p. 529). Fraud cases, such as that involving Diederik Stapel, show just how far a person might be willing to go when succumbing to such pressures (Stroebe, Postmes, & Spears, 2012; Derksen, 2021). Whether replication is really necessary and whether the problematic research practices mentioned above are due more to the present reward system, general human biases or an incorrect statistical or philosophical understanding are still open questions (Feest, 2019; Flis, 2019; Morawski, 2019). Given such uncertainties, it seems interesting to explore how research was undertaken in the past, when the current institutional conditions did not pertain—or at least, not yet fully. Stated differently: If the current replicability problem is related to recent research practices that have appeared as part of academic life in times of neoliberal capitalism and “big science,” then we might assume that replication worked differently in the past. Thus, in the present paper, I adopt a historical stance to reveal characteristics of nineteenth-century psychology experimental research practices and to describe the way research was replicated. The original experiments I present in this paper are well known, dating from 1860 to 1900, a period characterized by important changes in Europe, such as industrialization, workers movements, and the constitution of modern nation states. In this period, the Prussianmodel at German universities (Charle, 2004), offered a broad humanistic as well as a thorough experimental training in fields such as chemistry, physics, physiology, philosophy, and psychology, which was just emerging as an experimental sciencewith its own scientific community (Ash, 1980; Bringmann & Tweney, 1980; Rieber & Robinson, 2001). When the first replications were performed, neither young researchers nor senior professors suffered the pressure to “publish or perish,” and journals were not yet acting as gatekeepers taking decisions based on p-values. Often, researchers carried out years of patient experimenting before they published their results. Thus, what role did replication play within psychological research?