The power of effect size stabilization.

Abstract

Determining an appropriate sample size in psychological experiments is a common challenge, requiring a balance between maximizing the chance of detecting a true effect (minimizing false negatives) and minimizing the risk of observing an effect where none exists (minimizing false positives). A recent study proposes using effect size stabilization, a form of optional stopping, to define sample size without increasing the risk of false positives. In effect size stabilization, researchers monitor the effect size of their samples throughout the sampling process and stop sampling when the effect no longer varies beyond predefined thresholds. This study aims to improve our understanding of effect size stabilization properties. Simulations involving effect size stabilization are presented, with parametric modulation of the true effect in the population and the strictness of the stabilization rule. As previously demonstrated, the results indicate that optional stopping based on effect-size stabilization consistently yields unbiased samples over the long run. However, simulations also reveal that effect size stabilization does not guarantee the detection of a true effect in the population. Consequently, researchers adopting effect size stabilization put themselves at risk of increasing type 2 error probability. Instead of using effect-size stabilization procedures for testing, researchers should use them to reach accurate parameter estimates.