Analysis of the influence of communication between researchers on experiment replication

Abstract

The replication of experiments is a key undertaking in SE. Successful replications enable a discipline's body of knowledge to grow, as the results are added to those of earlier replications. However, replication is extremely difficult in SE, primarily because it is difficult to get a setting that is exactly the same as in the original experiment. Consequently, changes have to be made to the experiment to adapt it to the new site. To be able to replicate an experiment, information also has to be transmitted (usually orally and in writing) between the researchers who ran the experiment earlier and the ones who are going to replicate the experiment. This article examines the influence of the type of communication there is between experimenters on how successful a replication is. We have studied three replications of the same experiment in which different types of communication were used.