Conducting Laboratory Experiments Properly with Statistical Tools: An Easy Hands-On Tutorial

Abstract

This hands-on half-day tutorial consists of two sessions. Part~I covers the following topics: Preliminaries; Paired and two-sample t-tests, confidence intervals; One-way ANOVA and two-way ANOVA without replication; Familiwise error rate. Part~II covers the following topics: Tukey's HSD test, simultaneous confidence intervals; Randomisation test and randomised Tukey HSD test; What's wrong with statistical significance tests?; Effect sizes, statistical power; Topic set size design and power analysis; Summary: how to report your results. Participants should have some prior knowledge about the very basics of statistical significance testing and are strongly encouraged to bring a laptop with R already installed. They will learn how to design and conduct statistical significance tests for comparing the mean effectiveness scores of two or more systems appropriately, and to report on the test results in an informative manner.